How to Give Imperfect Automated Guidance to Learners: A Case-Study in Workplace Learning
How to Give Imperfect Automated Guidance to Learners: A Case-Study in Workplace Learning
复制标题
DOI：
10.1007/978-3-031-11644-5_1
复制
发表时间：
2022
期刊：
影响因子：
--
通讯作者：
J. Whitehill;Amitai Erfanian
中科院分区：
文献类型：
--
作者：
J. Whitehill;Amitai Erfanian
在模拟材料回收设施中的工人学习识别和操纵传送带上的物体的工作场所学习场景中，我们研究了机器学习（ML）助手的不完善指导如何影响学习者的体验和行为。具体来说，在一项随机实验中（参与者来自Amazon MTurk），我们改变了助手在检测不可回收物品时的假阳性（FP）和假阴性（FN）率，并评估了对学习者的表现，学习和信任的影响。我们还探索了一种软突出显示条件，即助理提供有关其自信程度的细粒度信息。我们发现的证据表明，FP/FN的权衡可以影响学习者的表现时，与助理合作，软突出显示条件可能会产生较少的信任，从学习者相比，其他条件。有初步的证据表明，工人的行为受到影响的FP/FN权衡他们分配的实验条件，即使在ML助理被删除。最后，在后续研究中，我们发现有证据表明，学习者会根据助理传达的细粒度信心值调整自己的行为。
In a workplace learning scenario in which workers in a simulated Material Recovery Facility learn to recognize and manipulate objects on conveyer belts, we studied how imperfect guidance from a machine learning (ML) assistant may impact learners’ experience and behaviors. Specifically, in a randomized experiment (participants from Amazon MTurk) we varied the assistant’s False Positive (FP) and False Negative (FN) rates in detecting non-recyclable objects and assessed the impact on learners’ performance, learning, and trust. We also explored a soft highlighting condition, whereby the assistant provides fine-grained information about how confident it is. We found evidence that the FP/FN trade-off can impact learners’ performance when working cooperatively with the assistant, and that the soft highlighting condition may generate less trust from learners compared to the other conditions. There was tentative evidence that workers’ behaviors were impacted by the FP/FN trade-off of their assigned experimental condition even after the ML assistant was removed. Finally, in a follow-up study () we found evidence that learners modulate their behaviors based on the fine-grained confidence values conveyed by the assistant.