Doctoral Dissertation Research: Intonational Cues in Emotional Contexts

通过语言传达的意义不仅取决于所说的话--声音、单词、短语--而且还取决于它是如何说的--音调、音量、节奏和语调的其他方面。在美式英语中，不同的语调模式被用来传达与话语的话语语境相关的意义，区分问题和断言，对话中说话者话轮的继续，或标记对提供新信息的词语的强调。从与语调相关的语音提示中破译语言意义是复杂的，因为相同的语音参数也是说话者情感表达的组成部分。话语意义和说话人情感这两个因素如何相互作用决定话语的语调形式，听话者如何从说话人的情绪状态的线索中理清语调线索对语言意义的影响？为了回答这个问题，这个项目调查了基于说话人情绪的语篇意义的语音编码的变化，以及听者如何使用关于说话人的情绪状态的信息来指导他们对语篇意义的解释。这个项目的第一阶段考察了来自语音产生实验的证据，这些证据在不同的设定情绪条件下产生的句子中诱导出特定的语调模式。第二阶段测试受试者对说话人情绪的感知如何影响他们处理语言语调范畴和理解语篇意义的方式。这一研究结果将有助于揭示语言语境和说话人情感在语调产生和感知过程中的相互作用。本文采用美国英语语调的自动分段-韵律模型，其中语调的语音实现来自于将单词分组为韵律短语的语音结构，以及标记韵律短语边缘(边界声调)和短语突出度(音高重音)的语调特征。本研究主要考察情绪如何影响八种具有不同语用意义的短语尾调“调”(如降调或升调)的产生。曲调是用模仿范例来引出的。受试者在四种设定情绪的情况下，听到一个模型句子的曲调，然后在一个与曲调意义一致的语篇语境中，对一个新句子产生相同的曲调。用贝叶斯混合效应回归方法模拟了情绪对语调声学相关的影响。在一系列知觉和理解实验中，来自生产实验的录音话语的子集被用作刺激。曲调之间的知觉辨别作为设定情绪的函数使用AXB辨别力进行测试，并且是一项自由分类任务，听者根据说话者推断的交际目标(例如，表达惊讶、做出断言)对话语进行分组。对曲调意义的理解是通过一组结构化的问题来测试的，这些问题探索听者对每个曲调的语用判断作为设定的情绪的功能。这个项目对语调形式和意义的语言学理论做出了贡献，对说话人的情绪作为语调产生和感知变化的来源有了新的见解，并为未来的语调和情绪研究贡献了一个新的语音记录语料库-英语语料库。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The meaning conveyed through language depends not only on what is said--sounds, words, phrases--but also on how it is said--pitch, loudness, tempo, and other dimensions of intonation. In American English, distinct intonation patterns are used to convey meaning related to the discourse context of an utterance, distinguishing questions from assertions, the continuation of a speaker's turn in a dialogue, or marking emphasis on words that contribute new information. Decoding linguistic meaning from phonetic cues related to intonation is complicated by the fact that the same phonetic parameters are also integral to the speaker’s expression of emotion. How do these two factors—discourse meaning and speaker emotion—interact in determining the intonational form of an utterance, and how do listeners disentangle intonational cues to linguistic meaning from cues to the speaker's emotional state? To answer this question, this project investigates variation in the phonetic encoding of discourse meaning based on speaker emotion, and how listeners use information about a speaker’s emotional state to guide their interpretation of discourse meaning. Phase I of this project examines evidence from speech production experiments that elicit specific intonation patterns in sentences produced under different conditions of enacted emotion. Phase II tests how listeners' perception of speaker emotion affects how they process linguistic intonational categories and the interpretation of discourse meaning. The findings from this study will shed light on the interaction between linguistic context and speaker emotion in the production and perception of intonation. This work adopts the Autosegmental-Metrical model of American English intonation, in which the phonetic implementation of intonation derives from phonological structures that group words into prosodic phrases, and tone features that mark prosodic phrase edges (boundary tones) and phrasal prominence (pitch accents). The investigation focuses on how enacted emotion affects the production of eight phrase-final intonation “tunes” (e.g., falling pitch or rising pitch) that have distinct pragmatic meaning. Tunes are elicited using an imitation paradigm. Participants hear a tune on a model sentence, and then produce the same tune on a new sentence presented in a discourse context congruent with the tune meaning, under four conditions of enacted emotion. The effect of emotion on acoustic correlates of intonation is modeled using Bayesian mixed-effects regression. A subset of recorded utterances from the production experiment are used as stimuli in a series of perception and comprehension experiments. Perceptual discrimination among tunes is tested as a function of enacted emotion using AXB discrimination, and a free classification task where listeners group utterances according to the speaker's inferred communicative goal (e.g., to express surprise, make an assertion). Comprehension of tune meaning is tested through a structured set of questions probing listeners’ pragmatic judgments for each tune as a function of the enacted emotion. This project contributes to linguistic theories of intonational form and meaning, with new insights into speaker emotion as a source of variation in intonation production and perception, and contributes a new corpus of speech recordings, the English IntEX Corpus, for future research on intonation and emotion.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.