Collaborative Research: CCRI: New: reVISit: Scalable Empirical Evaluation of Interactive Visualizations

该项目通过开发基础设施，允许研究人员进行功能丰富的在线实验，进一步促进了我们对数据可视化的理解。数据可视化在医学、生物或智能等各种领域的数据驱动的发现中至关重要。可视化也经常用于交流，例如报纸或政府机构。然而，很难知道哪种可视化技术更适合特定的数据集和任务。某些可视化技术能让人们更有效地或更正确地做出判断吗？一种特定的技术会带来更多的洞察力，还是会比其他技术带来更多样化的洞察力？哪一种技术更好用，还是更多才多艺？一种技术比另一种更容易学吗？在发展熟悉感之后，一项技术会更好吗？这些都是可视化社区需要回答的问题，以发展对数据可视化的严格理解。有了这些问题的答案，我们就可以做出更好的可视化设计选择，制定更好的可视化建议，并允许我们根据受众和情况定制可视化。该项目将开发测试基础设施，以便科学家可以在大规模的基于网络的实验中有效地提出这些问题，参与者不同，反映了可视化的目标受众。该团队将开发Reviser Infrastructure，这是一套模块化但兼容的工具，用于设计和调试交互式可视化技术的在线研究，为研究有效地设计培训材料，从参与者那里获得自由形式的口头回应，并提供用于分析此类数据的高级工具。拟议的重访基础设施开发解决了可视化研究中的一个关键瓶颈：我们如何更好、更有效地对可视化技术进行实证评估？重访基础设施旨在使交互式可视化技术的评估民主化，这是一个一直未得到充分探索的领域，部分原因是创建复杂的在线实验所需的高技术负担和技能。该项目的主要创新是：(1)与在线众包研究环境兼容的灵活研究创建和仪表化数据收集的软件基础设施，包括交互来源、见解和原理。(2)将结果数据整理成与现成分析工具兼容的格式的软件基础设施，以及分析这些可用于试点、质量控制和分析使用类型、洞察力、Rational和性能的不同数据流的高级软件基础设施。这些方法将允许可视化研究人员收集关于不同交互式可视化技术优点的经验证据。它将使研究人员了解不同技术支持的洞察类型，揭示用户可能采取的不同分析策略。最终，这些方法将使更多的可视化研究人员能够使用众包进行比以前更广泛的实验。这一奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This project furthers progress in our understanding of data visualization by developing infrastructure to allow researchers to conduct feature-rich online experiments. Data visualizations are crucial in data driven discovery in fields as diverse as medicine, biology, or intelligence. Visualization is also often used in communication, for example by newspapers or government agencies. However, it is difficult to know which visualization techniques are better for particular datasets and tasks. Can certain visualization techniques enable people to make judgements more efficiently, or more correctly? Does a particular technique lead to more insights, or to more diverse types of insights than others? Is one technique more enjoyable to use, or more versatile? Is one technique easier to learn than another one? Is a technique better after developing familiarity? These are the kinds of questions that the visualization community needs to answer to develop a rigorous understanding of data visualization. Having answers to these questions will allow us to make better visualization design choices, develop better recommendations for visualizations, and allow us to tailor visualizations to audiences and situations. This project will develop testing infrastructure, so that scientists can ask these questions efficiently in large scale web-based experiments, with diverse participants that reflect the intended audience of visualizations. The team will develop the reVISit Infrastructure, a suite of modular but compatible tools to design and debug online studies of interactive visualization techniques, to efficiently design training material for the study, to elicit free-form, spoken responses from participants, and to provide advanced tools for analyzing such data. The proposed reVISit infrastructure development addresses a critical bottleneck in visualization research: how can we better and more efficiently empirically evaluate visualization techniques? The reVISit infrastructure aims to democratize evaluation of interactive visualization techniques, an area that has been under-explored, due in part to the high technical burden and skills required to create complex online experiments. The key innovations of this project are: (1) Software infrastructure for flexible study creation and instrumented data collection, including interaction provenance, insights, and rationales, compatible with online crowdsourced study contexts. (2) Software infrastructure to wrangle the results data into formats compatible with off-the-shelf analysis tools, and advanced software infrastructure to analyze these diverse data streams that can be used for piloting, quality control, and analyzing usage types, insights, rational, and performance. These methods will allow visualization researchers to gather empirical evidence about the merits of different interactive visualization techniques. It will allow researchers to understand the types of insights that different techniques support, revealing diverging analysis strategies users may take. Ultimately, these methods will enable a wider set of visualization researchers to run a much broader range of experiments using crowdsourcing than before.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.