CAREER: Behavior-Driven Testing of Big Data Exploration Tools

职业：大数据探索工具的行为驱动测试

• 批准号: 2141506
• 负责人: Leilani Battle
• 金额: $ 57.05万
• 依托单位: University of Washington
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2027-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2141506&HistoricalAwards=false
• 关键词: CAREER; Behavior; Driven; Testing; Big

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Companies, governments, and institutions all over the world use massive datasets to make many decisions that impact our daily lives, such as how climate change is addressed, who is protected from COVID-19, or which investments to prioritize to maximize a company’s growth. However, big data is only valuable when it can provide useful insights, which analysts often seek to extract from data visualizations. Much like mastering a new recipe, it takes effort and skill to process data and design effective visualizations, and analysts are increasingly turning to computational tools to support their efforts to visualize massive data and process it into a form suitable for consumption. However, evaluating these tools is challenging, both because they are used for a wide variety of problems and by a wide variety of people, and standard tool benchmarks are unequipped to handle these variations. The vision for this project is to develop better ways to evaluate tools as they are used in the wild; if we can automate the way we evaluate data exploration tools, then we can automatically test new tools as soon as they are created, tune them to real workloads, and help analysts be more efficient and effective at generating insights. To this end, the research team will develop automated testing software that can determine: (1) whether a data exploration tool is capable of helping someone achieve the particular goals they have in exploring their data; and (2) what problems these tools and evaluation methods may introduce to the data exploration process. The team will also work with leading visualization researchers and software companies to fine-tune the software and maximize its impact, and develop new programs to help students learn fundamental visualization and research skills.To make the envisioned software feasible, the research objective of this project is to formally specify an analyst’s goals in exploring a dataset, and to measure whether a given system helps or hinders an analyst’s ability to achieve these specified data exploration goals. During data exploration, analysts visually and interactively query their data to help their organization make informed decisions. The research will be conducted in three phases. Phase 1 will theoretically and programmatically define a person’s exploration intent at different granularities (e.g., goals, sub-tasks, interaction patterns). Phase 2 will integrate foundational theory in HCI with AI path planning methods to generate a valid sequence of user interactions that achieve a programmatically defined intent. Phase 3 will extend the models from Phase 2 to develop customizable performance testing software that can simulate how people alternate between goal-directed interactions (i.e., following a planned sequence) and open-ended interactions (i.e., exploring alternative analyses). The research team will implement the framework as an open source platform so others can use the findings to evaluate their own systems and data exploration use cases.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项全部或部分由2021年美国救援计划法案（Public Law 117-2）资助。世界各地的公司、政府和机构使用大量数据集来做出许多影响我们日常生活的决策，例如如何应对气候变化，保护谁免受COVID-19的影响，或者优先考虑哪些投资以最大限度地提高公司的增长。然而，大数据只有在能够提供有用的见解时才有价值，而分析师往往试图从数据可视化中提取这些见解。就像掌握一个新配方一样，处理数据和设计有效的可视化需要努力和技能，分析师越来越多地转向计算工具来支持他们可视化大量数据并将其处理成适合消费的形式。然而，评估这些工具是具有挑战性的，因为它们被用于各种各样的问题和各种各样的人，标准的工具基准不具备处理这些变化。该项目的愿景是开发更好的方法来评估工具，因为它们在野外使用;如果我们可以自动化评估数据探索工具的方式，那么我们可以在创建新工具时自动测试它们，将它们调整到真实的工作负载，并帮助分析师更有效地生成见解。为此，研究团队将开发自动化测试软件，以确定：（1）数据探索工具是否能够帮助人们实现他们在探索数据时的特定目标;以及（2）这些工具和评估方法可能会给数据探索过程带来什么问题。该团队还将与领先的可视化研究人员和软件公司合作，对软件进行微调，使其影响最大化，并开发新的程序，帮助学生学习基本的可视化和研究技能。为了使设想的软件可行，该项目的研究目标是正式指定分析师在探索数据集时的目标，并测量给定的系统是否有助于或阻碍分析师实现这些指定的数据探索目标的能力。在数据探索过程中，分析师以可视化和交互式方式查询数据，以帮助其组织做出明智的决策。研究将分三个阶段进行。阶段1将在理论上和程序上定义一个人在不同粒度上的探索意图（例如，目标、子任务、交互模式）。第二阶段将把HCI中的基础理论与AI路径规划方法相结合，以生成有效的用户交互序列，从而实现编程定义的意图。第三阶段将扩展第二阶段的模型，开发可定制的性能测试软件，可以模拟人们如何在目标导向的交互（即，遵循计划的顺序）和开放式交互（即，探索替代分析）。该研究团队将把该框架作为一个开源平台来实施，这样其他人就可以使用研究结果来评估他们自己的系统和数据探索用例。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估来支持。