RI: Small: Incremental Speech Processing for Rapid Dialogue

该项目开发增量语言处理技术，使口语对话系统以一种更高度互动、更高效、更像人类的方式进行交流。大多数之前的对话系统都采用了严格的轮流机制，即每次只有一个人说话，在说话者说完之前不会试图理解或回应语音，与人与人之间的对话相比，系统响应的总体延迟时间很高。这导致系统无法提供一系列快速和重叠的响应，而人类对话者经常使用这些响应来实现有效和成功的沟通过程，包括后台通道、中断、协作完成、澄清和其他快速响应。这个项目是一个计算和实证研究，研究系统如何评估它自己对正在进行的用户语音的增量理解，从而指导它的战略决策，以启动如此快速和重叠的响应。在两个快节奏互动对话游戏的背景下，研究了实现这一决策的特征表示和响应策略。这些游戏都是经过精心挑选的，以支持对增量响应策略的客观评估和促进大规模数据收集的有趣玩法。所得到的计算模型可以改进一系列对话系统的会话技巧，这些对话系统不仅包括面向游戏的系统，而且还包括诸如智能辅导和培训系统、信息访问系统和娱乐应用等实际应用。这个项目的第二个产品是一个带注释的人类和人类系统对话数据的语料库，供其他研究人员使用。第三个产品是将相关软件整合到一个公共分发的工具包中，用于构建对话系统，支持进一步的研究和教育。

This project develops incremental language processing techniques to enable spoken dialogue systems to communicate in a way that is more highly interactive, more efficient, and more human-like. Most previous dialogue systems have employed a strict turn-taking regime in which one person speaks at a time, no attempt is made to understand or respond to speech until the speaker finishes speaking, and the overall latency in system responses is high in comparison to human-human conversation. This results in systems that are unable to provide a range of rapid and overlapping responses that human interlocutors frequently use to achieve an efficient and successful communication process, including back-channels, interruptions, collaborative completions, clarifications, and other rapid responses. This project is a computational and empirical investigation into how a system's assessment of its own incremental understanding of ongoing user speech can guide its strategic decisions to initiate such rapid and overlapping responses. The feature representations and response policies that can implement this decision-making are studied in the context of two fast-paced interactive dialogue games. These games are carefully chosen to support objective evaluation of incremental response strategies and fun gameplay that facilitates large-scale data collection.The resulting computational models may improve the conversational skills of a range of dialogue systems, including not only game-oriented systems but also practical applications such as intelligent tutoring and training systems, information access systems, and entertainment applications. A second product of this project is an annotated corpus of human-human and human-system dialogue data for use by other researchers. A third product is the incorporation of relevant software into a publicly distributed toolkit for building dialogue systems, supporting further research and education.