Crowdsourcing the Perception of Machine Teaching
Crowdsourcing the Perception of Machine Teaching
复制标题
DOI：
10.1145/3313831.3376428
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Jonggi Hong;Kyungjun Lee;June Xu;Hernisa Kacorri
中科院分区：
文献类型：
--
作者：
Jonggi Hong;Kyungjun Lee;June Xu;Hernisa Kacorri
可教界面可以通过明确提供相关的培训示例，使最终用户能够调整机器学习系统以适应其特有的特征和环境。在促进控制的同时，它们的有效性可能会因缺乏专业知识或误解而受到阻碍。我们通过在Amazon Machine Turk中部署移动可教学试验床，调查用户可能如何概念化、体验和反思他们在机器教学中的参与。使用基于性能的支付方案，机械特工队(N=100)被要求通过在其环境中拍摄的几个快照来实时训练、测试和重新训练健壮的识别模型。我们发现，参与者在他们的例子中融入了多样性，从平行到人类如何识别与大小、视点、位置和照明无关的对象。他们的许多误解与一致性和推理模型能力有关。由于测试中的变异和边缘情况有限，他们中的大多数人在第二次培训尝试时不会改变策略。
Teachable interfaces can empower end-users to attune machine learning systems to their idiosyncratic characteristics and environment by explicitly providing pertinent training examples. While facilitating control, their effectiveness can be hindered by the lack of expertise or misconceptions. We investigate how users may conceptualize, experience, and reflect on their engagement in machine teaching by deploying a mobile teachable testbed in Amazon Mechanical Turk. Using a performance-based payment scheme, Mechanical Turkers (N=100) are called to train, test, and re-train a robust recognition model in real-time with a few snapshots taken in their environment. We find that participants incorporate diversity in their examples drawing from parallels to how humans recognize objects independent of size, viewpoint, location, and illumination. Many of their misconceptions relate to consistency and model capabilities for reasoning. With limited variation and edge cases in testing, the majority of them do not change strategies on a second training attempt.