PaTAT: Human-AI Collaborative Qualitative Coding with Explainable Interactive Rule Synthesis
PaTAT: Human-AI Collaborative Qualitative Coding with Explainable Interactive Rule Synthesis
复制标题
DOI：
10.1145/3544548.3581352
复制
发表时间：
2023-04
期刊：
影响因子：
--
通讯作者：
Simret Araya Gebreegziabher;Zheng Zhang;Xiaohang Tang;Yihao Meng;Elena L. Glassman;Toby Jia-Jun Li
中科院分区：
文献类型：
--
作者：
Simret Araya Gebreegziabher;Zheng Zhang;Xiaohang Tang;Yihao Meng;Elena L. Glassman;Toby Jia-Jun Li
多年来，人工智能辅助数据注释的任务已经取得了显着的进步。然而，一种特定类型的注释任务，即在主题分析期间执行的定性编码，具有使有效的人类-人工智能协作变得困难的特征。通过形成性研究，我们设计了PaTAT，这是一种新的AI工具，它使用交互式程序合成方法，在用户注释数据时实时学习用户注释代码的灵活和表达模式。为了适应主题分析的模糊性，不确定性和迭代性，使用用户可解释的模式允许用户理解和验证系统所学到的内容，进行直接修复，并轻松修改，拆分或合并先前注释的代码。这种新方法除了促进人工智能模型的“学习”外，还有助于人类用户学习数据特征并形成新的理论。在实验室用户研究中评估了PaTAT的有用性和有效性。
Over the years, the task of AI-assisted data annotation has seen remarkable advancements. However, a specific type of annotation task, the qualitative coding performed during thematic analysis, has characteristics that make effective human-AI collaboration difficult. Informed by a formative study, we designed PaTAT, a new AI-enabled tool that uses an interactive program synthesis approach to learn flexible and expressive patterns over user-annotated codes in real-time as users annotate data. To accommodate the ambiguous, uncertain, and iterative nature of thematic analysis, the use of user-interpretable patterns allows users to understand and validate what the system has learned, make direct fixes, and easily revise, split, or merge previously annotated codes. This new approach also helps human users to learn data characteristics and form new theories in addition to facilitating the “learning” of the AI model. PaTAT’s usefulness and effectiveness were evaluated in a lab user study.