EAGER: Preliminary Explorations for the Development of Responsive Prosodic Behaviors for Interactive Systems

EArly的探索性研究资助的动机是口语互动作为人们与自主智能系统互动的一种方式的吸引力。 当系统与人交谈时，语音不仅可以传达词汇信息，还可以传达Meta信息，例如信息是否需要立即关注或只是背景，系统对用户的目标和情况的理解程度，以及系统是否需要继续提供信息或暂时完成。在人与人的对话中，这些元信息主要通过韵律（语调）来传达：话语中音高、能量、速率和时间的微妙变化。 不幸的是，如果不使用预先录制的提示或手工制作的合成话语，这种代理的表达能力今天是不可用的，对于在通信需求的可能配置无法提前完全预测的情况下运行的系统来说，这两种方式都不够灵活。因此，需要使口语技术更好地支持机器人和其他系统的交互。 韵律建模的进展也可以告知语言教师和沟通教师，以及处理沟通障碍的健康从业者。该项目侧重于在人机对话中使用韵律的目标的两个初步步骤：首先用不同的语音合成模型进行实验，以确定哪种模型最能支持韵律上适应良好的话语的实现，并确定需要在何处进行改进，无论是引擎本身还是训练数据收集过程;以及第二，引导数据收集和分析，以确定需要测量和表示上下文的哪些方面，以支持适当的韵律选择。

This EArly Grant for Exploratory Research is motivated by the attractiveness of spoken interaction as a way for people to interact with autonomous intelligent systems. When systems talk to people, speech can convey not only lexical information but also meta information, such as whether the information requires immediate attention or is just background, how well the system understands the user's goals and situation, and whether the system needs to continue providing information or is done for the moment. In human-human dialog such meta-information is mostly conveyed by prosody (intonation): subtle variations in the pitch, energy, rate and timing within utterances. Unfortunately this sort of expressiveness for agents is not today available without using pre-recorded prompts or hand-crafted synthesized utterances, neither of which is flexible enough for systems operating in contexts where the possible configurations of communication needs are not fully predictable ahead of time. Thus there is a need to make spoken language technology better support interaction for robots and other systems. Advances in prosody modeling can also inform language teachers and communication teachers, and health practitioners who deal with communications disorders.This project focuses on two preliminary steps to the goal of using prosody in human-computer dialog: first experimentation with different models of speech synthesis to determine which can best support the realization of prosodically well-adapted utterances, and to determine where improvements need to be made, whether to the engine itself or to the training-data collection procedures; and second a pilot data collection and analysis to determine which aspects of context need to be measured and represented to support appropriate prosodic choices.