STIMULATE: Modeling and Automatic Labeling of Hidden Word- Level Events in Spontaneous Speech

大多数当前的NLP技术期望类似于阅读或约束语音的输入。 当应用于自发言语时，这种技术遇到两个严重的困难。首先，自发言语包含与输入的非命题方面相关的表面现象，如不流利和话语标记语。 第二，自发语音缺乏明显的标点符号分割成有意义的单位输入。 对于有效的NLP，这些现象应该在输入中明显标记;然而，当前的语音识别器只能产生原始的单词序列。 该项目的目标是增强语音识别模型，以输出为这些现象注释的单词序列，称为“隐藏单词级事件”（HWE）。开发新模型以允许 识别HWE与以下事件同时发生 单词识别 HWE识别基于声学和语言模型的组合，扩展了当前系统中的标准组件。 新模型还捕捉了HWE的韵律特征，包括语调和 持续时间模式。 韵律信息相结合的统计语言模型描述HWE的分布有关的词汇和句法单位。研究结果将显著提高我们自动处理自发语音的能力，同时也将进一步加深我们对自发语音的基本理解。

Most current NLP techniques expect input resembling read or constrained speech. When applied to spontaneous speech, such techniques encounter two serious difficulties. First, spontaneous speech contains surface phenomena relating to non-propositional aspects of the input, such as disfluencies and discourse markers. Second, spontaneous speech lacks overt punctuation for segmenting the input into meaningful units. For effective NLP, such phenomena should be overtly marked in the input; current speech recognizers, however, produce only a raw sequence of words. The goal of this project is to augment speech recognition models to output word sequences annotated for these phenomena, termed "Hidden Word-Level Events" (HWEs). New models are developed to allow recognition of HWEs to occur in tandem with word recognition. HWE recognition is based on a combination of acoustic and language models, extending the standard components found in current systems. The new models also capture prosodic characteristics of HWEs, including intonation and duration patterns. The prosodic information is combined with statistical language models describing the distribution of HWEs in relation to lexical and syntactic units. Results should significantly enhance our ability to process spontaneous speech automatically; the research will also further our basic understanding of spontaneous speech.