Learning Human-Compatible Representations for Case-Based Decision Support
Learning Human-Compatible Representations for Case-Based Decision Support
复制标题
DOI：
10.48550/arxiv.2303.04809
复制
发表时间：
2023-03
期刊：
影响因子：
--
通讯作者：
Han Liu;Yizhou Tian;Chacha Chen;Shi Feng;Yuxin Chen;Chenhao Tan
中科院分区：
文献类型：
--
作者：
Han Liu;Yizhou Tian;Chacha Chen;Shi Feng;Yuxin Chen;Chenhao Tan
基于算法的案例决策支持提供了帮助人类理解预测标签的示例，并帮助人类进行决策任务。尽管监督学习的表现很有前景，但由监督模型学习的表征可能与人类的直觉不太一致：模型认为相似的例子可能被人类视为不同的例子。因此，它们在基于案例的决策支持方面的有效性有限。在这项工作中，我们将度量学习的思想与监督学习结合起来，以检查对齐对有效决策支持的重要性。除了实例级标签之外，我们还使用人类提供的三重判断来学习与人类兼容的以决策为中心的表示。通过在多个分类任务中使用合成数据和人类受试者实验，我们证明了这种表征比仅为分类优化的表征更符合人类感知。人类相容表征识别出被人类认为更相似的最近邻居，并允许人类做出更准确的预测，从而大大提高了人类的决策准确性（蝴蝶与飞蛾分类的准确率为17.8%，肺炎分类的准确率为13.2%）。
Algorithmic case-based decision support provides examples to help human make sense of predicted labels and aid human in decision-making tasks. Despite the promising performance of supervised learning, representations learned by supervised models may not align well with human intuitions: what models consider as similar examples can be perceived as distinct by humans. As a result, they have limited effectiveness in case-based decision support. In this work, we incorporate ideas from metric learning with supervised learning to examine the importance of alignment for effective decision support. In addition to instance-level labels, we use human-provided triplet judgments to learn human-compatible decision-focused representations. Using both synthetic data and human subject experiments in multiple classification tasks, we demonstrate that such representation is better aligned with human perception than representation solely optimized for classification. Human-compatible representations identify nearest neighbors that are perceived as more similar by humans and allow humans to make more accurate predictions, leading to substantial improvements in human decision accuracies (17.8% in butterfly vs. moth classification and 13.2% in pneumonia classification).