HRI: Collaborative Proposal: Incremental Wizard Ablation: A Novel WOz Paradigm for Learning, Testing and Evaluating Human-Machine Dialog Strategies from Parameterized Corpora

自动电话对话系统不成比例地依赖于将语音信号准确转录成可读文本。当系统对呼叫者话语的自动语音转录(ASR)的置信度较低时，典型的对话策略要求系统重复其最佳猜测，并要求确认。这会导致不自然的互动和不满意的呼叫者。目前的项目专注于通过自动从对比语料库中学习并比较结果来开发更好的对话策略，以满足当前的ASR能力。使用一种新的方法，向导消融，模拟的人体系统对话被收集，这些对话以受控的方式变化。这个测试床应用程序是纽约市安德鲁·海斯克尔有声读物和盲文图书馆的自动化读者顾问，其复杂性有限，潜在的社会效益也很广泛。驱除巫师的动机是，如果人类的沟通渠道被限制为更像机器的沟通渠道，那么需要研究人类将使用的解决问题的策略。在传统的绿野仙踪研究中，毫无戒心的用户与人类巫师在屏幕后互动，从而提供关于人类与(他们认为是)机器互动方式的数据。与传统向导不同，烧蚀向导仅限于看到系统对话管理器的ASR输入。在进一步的消融条件下，向导必须从系统使用的曲目中选择动作，但可以自由组合它们。巫师互动的图书借阅场景被设计成现实的，海斯克尔图书馆的赞助人参与了研究。收集到的对话框将提供给社区。

Automated telephone dialog systems rely disproportionately on accurate transcription of the speech signal into readable text. When the system has low confidence in the automatic speech transcription(ASR) of a caller's utterance, a typical dialog strategy requires the system to repeat its best guess, and ask for confirmation. This leads to unnatural interactions and dissatisfied callers. The current project focuses on developing better dialog strategies given current ASR capabilities by learning automatically from contrastingcorpora, and comparing the results. Using a novel methodology, wizard ablation, simulated human-system dialogs are collected that vary in controlled ways. The testbed application, an Automated Readers Advisor for New York City's Andrew Heiskell Talking Book and Braille Library, has appropriately limited complexity, andpotentially broad social benefit.The motivation for wizard ablation is that research is needed into the problem-solving strategies humans would use if the human communication channel were restricted to be more like a machine's. In conventional wizard-of-oz studies, unsuspecting users interact with human wizards "behind-the-screen", thus providing data on the way humans interact with (what they believe to be) machines. Unlike a conventional wizard, an ablated wizard is restricted to seeing the ASR input to the system dialog manager. Under a further ablationcondition, the wizard must choose actions from the repertoire that the system uses, but can combine them freely. The book-borrowing scenarios for the wizard interactions have been designed to be realistic, and Heiskell Library patrons participate in the studies. The collected dialogs will be made available to the community.