Incremental Wizard Ablation: A Novel WOz Paradigm for Learning, Testing and Evaluating Human-Machine Dialogue using Parameterized Corpora

增量向导消融：使用参数化语料库学习、测试和评估人机对话的新型 WOz 范式

• 批准号: 0744904
• 负责人: Susan Epstein
• 金额: $ 20.57万
• 依托单位: CUNY Hunter College
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2007
• 资助国家: 美国
• 起止时间: 2007-10-01 至 2012-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0744904&HistoricalAwards=false
• 关键词: Incremental; Wizard; Ablation; Novel; WOz

Automated telephone dialog systems rely disproportionately on accurate transcription of the speech signal into readable text. When the system has low confidence in the automatic speech transcription(ASR) of a caller's utterance, a typical dialog strategy requires the system to repeat its best guess, and ask for confirmation. This leads to unnatural interactions and dissatisfied callers. The current project focuses on developing better dialog strategies given current ASR capabilities by learning automatically from contrastingcorpora, and comparing the results. Using a novel methodology, wizard ablation, simulated human-system dialogs are collected that vary in controlled ways. The testbed application, an Automated Readers Advisor for New York City's Andrew Heiskell Talking Book and Braille Library, has appropriately limited complexity, andpotentially broad social benefit.The motivation for wizard ablation is that research is needed into the problem-solving strategies humans would use if the human communication channel were restricted to be more like a machine's. In conventional wizard-of-oz studies, unsuspecting users interact with human wizards "behind-the-screen", thus providing data on the way humans interact with (what they believe to be) machines. Unlike a conventional wizard, an ablated wizard is restricted to seeing the ASR input to the system dialog manager. Under a further ablationcondition, the wizard must choose actions from the repertoire that the system uses, but can combine them freely. The book-borrowing scenarios for the wizard interactions have been designed to be realistic, and Heiskell Library patrons participate in the studies. The collected dialogs will be made available to the community.

自动电话对话系统不成比例地依赖于将语音信号准确地转录成可读文本。当 系统在呼叫者话语的自动语音转录（ASR）中具有低置信度，典型的对话策略要求系统重复其最佳猜测并请求确认。这会导致不自然的交互和不满意的呼叫者。目前的项目重点是通过从对比语料库中自动学习，并比较结果，开发更好的对话策略。使用一种新的方法，向导消融，模拟人类系统的对话收集不同的控制方式。测试平台应用程序是纽约市安德鲁·海斯凯尔有声读物和盲文图书馆的自动读者顾问，它的复杂性受到了适当的限制，但潜在的社会效益却很广泛。消除巫师的动机是，如果人类的交流渠道被限制得更像机器的话，人类将使用什么样的解决问题的策略，这是需要研究的。 在传统的绿野仙踪研究中，毫无戒心的用户在“屏幕后”与人类巫师互动，从而提供了人类与（他们认为是）机器互动方式的数据。与传统向导不同，消融向导被限制为查看输入到系统对话管理器的ASR。在进一步的消融条件下，向导必须从系统使用的指令表中选择动作，但可以自由地将它们联合收割机组合。向导互动的图书借阅场景被设计得很现实，Heiskell图书馆的顾客也参与了研究。 收集的对话将提供给社区。