REU Supplement to Incremental Wizard Ablation: A Novel WOz Paradigm for Learning, Testing and Evaluating Human-Machine Dialogue using Parameterized Corpora

自动电话对话系统过分依赖于将语音信号准确地转录成可读的文本。当系统对呼叫者话语的自动语音转录（ASR）缺乏信心时，典型的对话策略要求系统重复其最佳猜测，并请求确认。这会导致不自然的交互和不满意的呼叫者。目前的项目侧重于通过自动学习对比语料库，并比较结果，开发更好的对话策略。使用一种新颖的方法，向导消融，模拟的人类系统对话被收集，以受控的方式变化。测试平台应用程序是纽约市Andrew Heiskell Talking Book和盲文图书馆的自动读者顾问，它具有适当的有限复杂性，并具有潜在的广泛社会效益。巫师消融的动机是，如果人类的沟通渠道被限制得更像机器，那么需要研究人类将使用的解决问题的策略。在传统的wizard-of-oz研究中，毫无戒心的用户与“幕后”的人类向导进行交互，从而提供有关人类与（他们认为是）机器交互方式的数据。与传统向导不同，精简向导仅限于查看系统对话框管理器中的ASR输入。在进一步消融条件下，向导必须从系统使用的操作表中选择操作，但可以自由地组合它们。巫师互动的借书场景被设计得很现实，海斯凯尔图书馆的顾客也参与了研究。收集到的对话将提供给社区。

Automated telephone dialog systems rely disproportionately on accurate transcription of the speech signal into readable text. When the system has low confidence in the automatic speech transcription (ASR) of a caller's utterance, a typical dialog strategy requires the system to repeat its best guess, and ask for confirmation. This leads to unnatural interactions and dissatisfied callers. The current project focuses on developing better dialog strategies given current ASR capabilities by learning automatically from contrasting corpora, and comparing the results. Using a novel methodology, wizard ablation, simulated human-system dialogs are collected that vary in controlled ways. The testbed application, an Automated Readers Advisor for New York City's Andrew Heiskell Talking Book and Braille Library, has appropriately limited complexity, and potentially broad social benefit. The motivation for wizard ablation is that research is needed into the problem-solving strategies humans would use if the human communication channel were restricted to be more like a machine's. In conventional wizard-of-oz studies, unsuspecting users interact with human wizards "behind-the-screen", thus providing data on the way humans interact with (what they believe to be) machines. Unlike a conventional wizard, an ablated wizard is restricted to seeing the ASR input to the system dialog manager. Under a further ablation condition, the wizard must choose actions from the repertoire that the system uses, but can combine them freely. The book-borrowing scenarios for the wizard interactions have been designed to be realistic, and Heiskell Library patrons participate in the studies. The collected dialogs will be made available to the community.