Doctoral Dissertation Research: Intonational Cues in Emotional Contexts

语言传达的意思不仅取决于所说的内容——声音、词语、短语——还取决于说话的方式——音高、响度、节奏和语调的其他方面。在美式英语中，不同的语调模式被用来传达与话语语境相关的意义，区分疑问句和断言，在对话中继续说话人的回合，或者在提供新信息的单词上做强调。从语调相关的语音线索解码语言意义是复杂的，因为同样的语音参数也是说话者情感表达的组成部分。话语意义和说话者情绪这两个因素在决定话语语调形式时是如何相互作用的？听者又是如何将语调线索从说话者的情绪状态线索中分离出来的？为了回答这个问题，本项目研究了基于说话者情绪的话语意义语音编码的变化，以及听者如何使用说话者情绪状态的信息来指导他们对话语意义的解释。这个项目的第一阶段研究了语音产生实验的证据，这些实验在不同的情绪状态下产生的句子中引出了特定的语调模式。第二阶段测试听者对说话者情绪的感知如何影响他们对语言语调类别的处理和话语意义的解释。本研究的结果将揭示语言语境和说话者情绪在语调产生和感知中的相互作用。本研究采用了美式英语语调的自段格律模型，其中语调的语音实现来源于将单词分组成韵律短语的语音结构，以及标记韵律短语边缘（边界音调）和短语突出（音高重音）的音调特征。本研究的重点是情绪如何影响八种具有不同语用意义的短语末调（如降调或升调）的产生。曲调是用模仿范式引出的。在四种情境下，参与者先听一个模型句子的曲调，然后在与曲调意义一致的话语语境中，在四种情境下，对一个新句子做出同样的曲调。利用贝叶斯混合效应回归模型建立了情绪对声调声学相关因素的影响。从生成实验中记录的话语子集被用作一系列感知和理解实验的刺激。通过使用AXB辨别和一个自由分类任务来测试音调之间的感知辨别，作为制定情感的功能，听者根据说话者推断的交际目标（例如，表达惊讶，做出断言）对话语进行分组。对曲调意义的理解是通过一组结构化的问题来测试听众对每个曲调的语用判断作为所制定的情感的功能。本项目对语调形式和意义的语言学理论做出了贡献，对说话者情绪作为语调产生和感知变化的来源有了新的认识，并为未来语调和情绪的研究提供了一个新的语音记录语料库——英语IntEX语料库。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The meaning conveyed through language depends not only on what is said--sounds, words, phrases--but also on how it is said--pitch, loudness, tempo, and other dimensions of intonation. In American English, distinct intonation patterns are used to convey meaning related to the discourse context of an utterance, distinguishing questions from assertions, the continuation of a speaker's turn in a dialogue, or marking emphasis on words that contribute new information. Decoding linguistic meaning from phonetic cues related to intonation is complicated by the fact that the same phonetic parameters are also integral to the speaker’s expression of emotion. How do these two factors—discourse meaning and speaker emotion—interact in determining the intonational form of an utterance, and how do listeners disentangle intonational cues to linguistic meaning from cues to the speaker's emotional state? To answer this question, this project investigates variation in the phonetic encoding of discourse meaning based on speaker emotion, and how listeners use information about a speaker’s emotional state to guide their interpretation of discourse meaning. Phase I of this project examines evidence from speech production experiments that elicit specific intonation patterns in sentences produced under different conditions of enacted emotion. Phase II tests how listeners' perception of speaker emotion affects how they process linguistic intonational categories and the interpretation of discourse meaning. The findings from this study will shed light on the interaction between linguistic context and speaker emotion in the production and perception of intonation. This work adopts the Autosegmental-Metrical model of American English intonation, in which the phonetic implementation of intonation derives from phonological structures that group words into prosodic phrases, and tone features that mark prosodic phrase edges (boundary tones) and phrasal prominence (pitch accents). The investigation focuses on how enacted emotion affects the production of eight phrase-final intonation “tunes” (e.g., falling pitch or rising pitch) that have distinct pragmatic meaning. Tunes are elicited using an imitation paradigm. Participants hear a tune on a model sentence, and then produce the same tune on a new sentence presented in a discourse context congruent with the tune meaning, under four conditions of enacted emotion. The effect of emotion on acoustic correlates of intonation is modeled using Bayesian mixed-effects regression. A subset of recorded utterances from the production experiment are used as stimuli in a series of perception and comprehension experiments. Perceptual discrimination among tunes is tested as a function of enacted emotion using AXB discrimination, and a free classification task where listeners group utterances according to the speaker's inferred communicative goal (e.g., to express surprise, make an assertion). Comprehension of tune meaning is tested through a structured set of questions probing listeners’ pragmatic judgments for each tune as a function of the enacted emotion. This project contributes to linguistic theories of intonational form and meaning, with new insights into speaker emotion as a source of variation in intonation production and perception, and contributes a new corpus of speech recordings, the English IntEX Corpus, for future research on intonation and emotion.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.