"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans
"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans
复制标题
DOI：
10.1145/3313831.3376873
复制
发表时间：
2020-01
期刊：
影响因子：
--
通讯作者：
Vivian Lai;Han Liu;Chenhao Tan
中科院分区：
文献类型：
--
作者：
Vivian Lai;Han Liu;Chenhao Tan
为了用机器学习模型支持人类决策，我们经常需要阐明模型中嵌入的模式，这些模式对人类来说是不明显的、未知的或违反直觉的。虽然现有的方法专注于在实时辅助下解释机器预测，但我们探索了模型驱动的教程，以帮助人类在训练阶段理解这些模式。我们认为这两个教程都有来自科学论文的指导方针，类似于当前的科学传播实践，并从训练数据中自动选择示例并进行解释。我们使用欺骗性评论检测作为测试平台，并进行大规模的随机人类受试者实验，以检查此类教程的有效性。我们发现，教程确实提高了人类的表现，有和没有实时援助。特别是，尽管深度学习提供了比简单模型上级的预测性能，但来自简单模型的教程和解释对人类更有用。我们的工作为以人为本的教程和解释提出了未来的方向，以实现人类和人工智能之间的协同作用。
To support human decision making with machine learning models, we often need to elucidate patterns embedded in the models that are unsalient, unknown, or counterintuitive to humans. While existing approaches focus on explaining machine predictions with real-time assistance, we explore model-driven tutorials to help humans understand these patterns in a train- ing phase. We consider both tutorials with guidelines from scientific papers, analogous to current practices of science communication, and automatically selected examples from training data with explanations. We use deceptive review detection as a testbed and conduct large-scale, randomized human-subject experiments to examine the effectiveness of such tutorials. We find that tutorials indeed improve human performance, with and without real-time assistance. In particular, although deep learning provides superior predictive performance than simple models, tutorials and explanations from simple models are more useful to humans. Our work suggests future directions for human-centered tutorials and explanations towards a synergy between humans and AI.