CRII: HCC: RUI: Visualization-Based Multimodal Data Analysis for Qualitative Research

该奖项全部或部分由《2021年美国救援计划法案》（公法117-2）资助。该项目的目标是建立基本的可视化分析策略，整合多模态数据和人在环机器学习技术，以促进和支持透明和可信赖的定性数据分析过程。定性研究人员收集和分析非数字数据，以了解人们的互动，意见和经验。定性研究中存在的潜在偏差是一个公认的问题，但在分析阶段增加透明度和可信度的研究有限。本研究通过文本分析和多模态数据提取的集成，为可视化在解决定性数据分析中的当前挑战方面发挥了新的作用。鉴于学术界普遍使用定性研究，缺乏透明度和验证的定性数据分析可能会产生深远的负面影响，如歧视性政策、次优患者护理和强化污名。因此，定性数据分析应严格进行，以产生可信和有意义的结果。所开发的技术将对数据分析、可视化和人机交互领域做出重大贡献。尽管定性数据分析软件取得了进步，但目前定性研究过程中存在三个关键困境：大量数据带来的认知负担，研究者引入的主观性和潜在偏见，以及包含重要非语言线索（如声调和面部表情）的多模态数据未得到充分利用。本提案的具体目标是确定和证明：(1)如何将文本分析技术与可视化和人类反馈相结合，以减轻认知负担和减少偏见；(2)音频特征的视觉总结如何促进非语言线索在定性分析中的结合。支持这些增强分析技术的开源可视化web工具将使用迭代的、以用户为中心的设计方法开发。将选择具有定性分析经验的潜在用户参加参与式设计会议、可用性测试和实地研究。实地研究将以镰状细胞病为基础，这是一个对患者进行污名化的常见话题。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2). The goal of this project is to establish fundamental visual analysis strategies that integrate multimodal data and human-in-the-loop machine learning techniques to promote and support a transparent and trustworthy qualitative data analysis process. Qualitative researchers collect and analyze non-numerical data to understand people's interactions, opinions, and experiences. The presence of potential bias in qualitative research is a well-recognized problem, but research to increase transparency and trustworthiness in the analysis phases have been limited. This research develops a novel role for visualizations in addressing current challenges in qualitative data analysis through the integration of text analysis and multimodal data extraction. Given the prevalent use of qualitative research in academia, qualitative data analysis without transparency and verification can have far-reaching negative impacts such as discriminating policies, suboptimal patient-care, and reinforced stigmas. Thus, qualitative data analysis should be conducted in a rigorous manner to yield trustworthy and meaningful results. The techniques developed will significantly contribute to the data analysis, visualization, and human-computer interaction fields.Despite advances in qualitative data analysis software, there are three key dilemmas in the current qualitative research process: the cognitive burden resulting from the vast amount of data, subjectivity and potential bias that are introduced by the researcher, and the underutilization of multimodal data containing important non-verbal cues such as vocal tones and facial expressions. The specific objective of this proposal is to identify and demonstrate: (1) how text analysis techniques can be combined with visualization and human feedback to alleviate cognitive burden and lessen bias; and, (2) how visual summaries of audio features can promote the incorporation of non-verbal cues in qualitative analysis. An open-source visualization webtool supporting these enhanced analysis techniques will be developed using an iterative, user-centered design methodology. Prospective users with qualitative analysis experience will be selected for a participatory design session, usability tests, and a field study. The field study will be grounded in the case of sickle cell disease, a topic in which stigmatization of patients is common.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.