Doctoral Dissertation Research: The Interaction of Pitch and Timing in the Perception of Prosodic Grouping

说话者将原本连续的语音流分成更小、更有意义的片段，其边缘用诸如停顿、速率变化和音调移动等可听提示来标记。韵律边界，作为这些片段的边缘和标记它们的线索，被认为对语言处理和口语习得起着至关重要的作用。虽然对标记边界的线索范围的记录已经取得了很大的进展，但关于听者在解释语流时如何理解这些线索的认知过程还不是很清楚。边界的信号包括来自时序和音调的多个线索。当前的韵律边界模型，例如在口语处理和文本到语音(TTS)系统中使用的模型，严重依赖定时提示。音调提示通常被认为仅支持计时提示，甚至被认为是多余的。然而，越来越多的证据表明，音调和时间在知觉中相互作用，包括言语和非言语研究证明了基于音调的知觉持续时间扭曲。本论文试图通过对通常独立测量的声学线索的知觉相互作用、整合和加权的实证工作，加强我们对边界知觉所涉及的心理过程的了解。这些相互关系的量化将通过反映主要使用客观时长测量的模型中缺乏的丰富的韵律结构来为口语处理中的边界检测和合成语音中的边界生成模型提供信息。与自然语音相比，合成语音具有更高的认知负荷(可理解性更低，召回率更低)，对于非母语人士、老年和/或听力障碍者或语言障碍者等人群来说尤其具有挑战性。众所周知，多余的声学线索可以在嘈杂的环境中提高语音的可理解性，并有可能减轻听者的认知负担。提高基于音调的提示的强度以促进边界处理可能会增加合成语音和基于TTS的辅助技术的可理解性和自然度。

Speakers break the otherwise continuous stream of their speech into smaller, meaningful segments, the edges of which are marked by audible cues such as pauses, rate changes and pitch movement. Prosodic boundaries, as these segment edges and the cues that mark them, are known to play a role critical to language processing and spoken language acquisition. While great progress has been made documenting the range of cues that mark boundaries, much is not understood about the cognitive processes listeners use to make sense of these cues in interpreting the speech stream. The signaling of a boundary includes multiple cues from timing and pitch. Current models of prosodic boundaries, such as are used in spoken language processing and text-to-speech (TTS) systems, rely heavily on timing cues. Pitch cues are typically considered to merely support timing cues, and are even considered redundant. However, growing evidence suggests that pitch and timing interact in perception, including speech and non-speech research demonstrating pitch-based distortions of perceived duration.This dissertation project seeks to enhance our knowledge of the psychological processes involved in boundary perception through empirical work on the perceptual interaction, integration and weighting of acoustic cues that have typically been measured independently. Quantifications of these interrelations will inform models of boundary detection in spoken language processing, and boundary generation in synthetic speech, by reflecting a richness of prosodic structure lacking in models using primarily objective duration measures. Synthetic speech has been shown to have a higher cognitive load (lower intelligibility, lower recall) than natural speech, and is especially challenging for populations such as non-native speakers, aging and/or hearing-impaired speakers, or those with language disorders. Redundant acoustic cues are known to increase speech comprehensibility in noisy conditions, and to potentially lighten the listeners' cognitive load. Increasing the strength of pitch-based cues that facilitate boundary processing may increase both the intelligibility and naturalness of synthetic speech and TTS-based assistive technologies.