Doctoral Dissertation Research: Intonational Cues in Emotional Contexts

通过语言传达的意义不仅取决于说了什么--声音、单词、短语--还取决于说的方式--音高、响度、克里思和语调的其他方面。在美式英语中，不同的语调模式被用来传达与话语语境相关的意义，区分疑问句和断言，在对话中继续说话者的回合，或者强调提供新信息的单词。从与语调相关的语音线索解码语言意义是复杂的，因为相同的语音参数也是说话者情感表达的组成部分。话语意义和说话人情感这两个因素在决定话语的语调形式时是如何相互作用的？听者又是如何将语调线索从说话人情感状态的线索中分离出来的？为了回答这个问题，本项目调查的语音编码的话语意义的基础上说话人的情绪变化，以及听众如何使用有关说话人的情绪状态的信息，以指导他们的话语意义的解释。这个项目的第一阶段研究言语产生实验的证据，这些实验引出了在不同情绪条件下产生的句子中的特定语调模式。第二阶段测试听者对说话人情感的感知如何影响他们对语言语调类别的处理和对话语意义的解释。本研究的发现将有助于揭示语言语境和说话者情感在语调产生和感知中的相互作用。这项工作采用了美国英语语调的Autosegmental-Metrical模型，其中语调的语音实现来自于将单词分组为韵律短语的语音结构，以及标记韵律短语边缘（边界音调）和短语突出（音高重音）的音调特征。调查的重点是如何制定的情绪影响生产的八个短语最后的语调“曲调”（例如，降调或升调）具有不同的语用意义。曲调是使用模仿范式引出的。受试者在四种情绪情境下，在一个典型句子上听到一个曲调，然后在一个与曲调意义一致的话语语境中产生一个新的句子。情绪对语调声学相关的影响采用贝叶斯混合效应回归建模。从生产实验中记录的话语的子集被用作一系列的感知和理解实验的刺激。曲调之间的感知区分被测试为使用AXB区分的所制定的情感的函数，以及自由分类任务，其中听者根据说话者的推断的交际目标（例如，表示惊讶，作出断言）。曲调意义的理解是通过一组结构化的问题来测试的，这些问题探讨了听者对每个曲调的语用判断，作为制定情感的函数。该项目有助于语调形式和意义的语言学理论，对说话者情感作为语调产生和感知变化的来源有了新的见解，并贡献了一个新的语音记录语料库，英语IntEX语料库，该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准。

The meaning conveyed through language depends not only on what is said--sounds, words, phrases--but also on how it is said--pitch, loudness, tempo, and other dimensions of intonation. In American English, distinct intonation patterns are used to convey meaning related to the discourse context of an utterance, distinguishing questions from assertions, the continuation of a speaker's turn in a dialogue, or marking emphasis on words that contribute new information. Decoding linguistic meaning from phonetic cues related to intonation is complicated by the fact that the same phonetic parameters are also integral to the speaker’s expression of emotion. How do these two factors—discourse meaning and speaker emotion—interact in determining the intonational form of an utterance, and how do listeners disentangle intonational cues to linguistic meaning from cues to the speaker's emotional state? To answer this question, this project investigates variation in the phonetic encoding of discourse meaning based on speaker emotion, and how listeners use information about a speaker’s emotional state to guide their interpretation of discourse meaning. Phase I of this project examines evidence from speech production experiments that elicit specific intonation patterns in sentences produced under different conditions of enacted emotion. Phase II tests how listeners' perception of speaker emotion affects how they process linguistic intonational categories and the interpretation of discourse meaning. The findings from this study will shed light on the interaction between linguistic context and speaker emotion in the production and perception of intonation. This work adopts the Autosegmental-Metrical model of American English intonation, in which the phonetic implementation of intonation derives from phonological structures that group words into prosodic phrases, and tone features that mark prosodic phrase edges (boundary tones) and phrasal prominence (pitch accents). The investigation focuses on how enacted emotion affects the production of eight phrase-final intonation “tunes” (e.g., falling pitch or rising pitch) that have distinct pragmatic meaning. Tunes are elicited using an imitation paradigm. Participants hear a tune on a model sentence, and then produce the same tune on a new sentence presented in a discourse context congruent with the tune meaning, under four conditions of enacted emotion. The effect of emotion on acoustic correlates of intonation is modeled using Bayesian mixed-effects regression. A subset of recorded utterances from the production experiment are used as stimuli in a series of perception and comprehension experiments. Perceptual discrimination among tunes is tested as a function of enacted emotion using AXB discrimination, and a free classification task where listeners group utterances according to the speaker's inferred communicative goal (e.g., to express surprise, make an assertion). Comprehension of tune meaning is tested through a structured set of questions probing listeners’ pragmatic judgments for each tune as a function of the enacted emotion. This project contributes to linguistic theories of intonational form and meaning, with new insights into speaker emotion as a source of variation in intonation production and perception, and contributes a new corpus of speech recordings, the English IntEX Corpus, for future research on intonation and emotion.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.