REU Supplement to Incremental Wizard Ablation: A Novel WOz Paradigm for Learning, Testing and Evaluating Human-Machine Dialogue using Parameterized Corpora

REU 对增量向导消融的补充：使用参数化语料库学习、测试和评估人机对话的新型 WOz 范式

• 批准号: 0849666
• 负责人: Susan Epstein
• 金额: $ 1.2万
• 依托单位: CUNY Hunter College
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2009
• 资助国家: 美国
• 起止时间: 2009-01-01 至 2010-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0849666&HistoricalAwards=false
• 关键词: REU; Supplement; Incremental; Wizard; Ablation

Automated telephone dialog systems rely disproportionately on accurate transcription of the speech signal into readable text. When the system has low confidence in the automatic speech transcription (ASR) of a caller's utterance, a typical dialog strategy requires the system to repeat its best guess, and ask for confirmation. This leads to unnatural interactions and dissatisfied callers. The current project focuses on developing better dialog strategies given current ASR capabilities by learning automatically from contrasting corpora, and comparing the results. Using a novel methodology, wizard ablation, simulated human-system dialogs are collected that vary in controlled ways. The testbed application, an Automated Readers Advisor for New York City's Andrew Heiskell Talking Book and Braille Library, has appropriately limited complexity, and potentially broad social benefit. The motivation for wizard ablation is that research is needed into the problem-solving strategies humans would use if the human communication channel were restricted to be more like a machine's. In conventional wizard-of-oz studies, unsuspecting users interact with human wizards "behind-the-screen", thus providing data on the way humans interact with (what they believe to be) machines. Unlike a conventional wizard, an ablated wizard is restricted to seeing the ASR input to the system dialog manager. Under a further ablation condition, the wizard must choose actions from the repertoire that the system uses, but can combine them freely. The book-borrowing scenarios for the wizard interactions have been designed to be realistic, and Heiskell Library patrons participate in the studies. The collected dialogs will be made available to the community.

自动电话对话系统不成比例地依赖于将语音信号准确地转录成可读文本。当系统对呼叫者话语的自动语音转录（ASR）具有低置信度时，典型的对话策略要求系统重复其最佳猜测，并要求确认。这会导致不自然的交互和不满意的呼叫者。目前的项目重点是通过从对比语料库中自动学习并比较结果，开发更好的对话策略。使用一种新的方法，向导消融，模拟人类系统的对话收集不同的控制方式。该测试平台应用程序是纽约市的Andrew Heiskell有声图书和盲文图书馆的自动读者顾问，具有适当有限的复杂性和潜在的广泛社会效益。巫师消融的动机是，如果人类的沟通渠道被限制得更像机器的话，人类需要研究解决问题的策略。在传统的绿野仙踪研究中，毫无戒心的用户在“屏幕后”与人类巫师互动，从而提供了人类与（他们认为是）机器互动方式的数据。与传统向导不同，消融向导被限制为查看输入到系统对话管理器的ASR。在进一步消融条件下，向导必须从系统使用的指令表中选择动作，但可以自由地将它们联合收割机组合。向导互动的图书借阅场景被设计得很现实，Heiskell图书馆的顾客也参与了研究。收集的对话将提供给社区。