Doctoral Dissertation Research: The Interaction of Pitch and Timing in the Perception of Prosodic Grouping

说话者将原本连续的语音流分成更小的、有意义的片段，这些片段的边缘由诸如停顿、速率变化和音高移动之类的听觉线索标记。韵律边界，作为这些片段边缘和标记它们的线索，被认为在语言处理和口语习得中起着至关重要的作用。虽然在记录标记边界的线索范围方面取得了很大进展，但对于听者在解释语音流时用来理解这些线索的认知过程，人们还不太了解。边界的信令包括来自定时和音高的多个提示。 目前的韵律边界模型，如口语处理和文本到语音（TTS）系统中使用的，严重依赖于时间线索。音高提示通常被认为仅仅支持定时提示，甚至被认为是冗余的。然而，越来越多的证据表明，音高和时间相互作用的知觉，包括语音和非语音研究表明，基于音高的失真perceived duration.This博士论文项目旨在提高我们的知识，通过对感知的相互作用，整合和加权的声学线索，通常是独立测量的经验工作，在边界知觉的心理过程。这些相互关系的量化将通知模型的边界检测在口语处理中，和合成语音中的边界生成，反映了丰富的韵律结构缺乏模型使用主要是客观的持续时间措施。合成语音已被证明比自然语音具有更高的认知负荷（较低的可懂度，较低的回忆），并且对于非母语人士，老年人和/或听力受损的说话者或语言障碍者等人群尤其具有挑战性。已知冗余声学线索在噪声条件下增加语音可理解性，并且潜在地减轻听者的认知负荷。增加有助于边界处理的基于音高的线索的强度可以增加合成语音和基于TTS的辅助技术的可懂度和自然度。

Speakers break the otherwise continuous stream of their speech into smaller, meaningful segments, the edges of which are marked by audible cues such as pauses, rate changes and pitch movement. Prosodic boundaries, as these segment edges and the cues that mark them, are known to play a role critical to language processing and spoken language acquisition. While great progress has been made documenting the range of cues that mark boundaries, much is not understood about the cognitive processes listeners use to make sense of these cues in interpreting the speech stream. The signaling of a boundary includes multiple cues from timing and pitch. Current models of prosodic boundaries, such as are used in spoken language processing and text-to-speech (TTS) systems, rely heavily on timing cues. Pitch cues are typically considered to merely support timing cues, and are even considered redundant. However, growing evidence suggests that pitch and timing interact in perception, including speech and non-speech research demonstrating pitch-based distortions of perceived duration.This dissertation project seeks to enhance our knowledge of the psychological processes involved in boundary perception through empirical work on the perceptual interaction, integration and weighting of acoustic cues that have typically been measured independently. Quantifications of these interrelations will inform models of boundary detection in spoken language processing, and boundary generation in synthetic speech, by reflecting a richness of prosodic structure lacking in models using primarily objective duration measures. Synthetic speech has been shown to have a higher cognitive load (lower intelligibility, lower recall) than natural speech, and is especially challenging for populations such as non-native speakers, aging and/or hearing-impaired speakers, or those with language disorders. Redundant acoustic cues are known to increase speech comprehensibility in noisy conditions, and to potentially lighten the listeners' cognitive load. Increasing the strength of pitch-based cues that facilitate boundary processing may increase both the intelligibility and naturalness of synthetic speech and TTS-based assistive technologies.