Audio-visual prosody of whispered and semi-whispered speech

耳语和半耳语语音的视听韵律

• 批准号: 426673330
• 负责人: Dr. Marzena Zygis
• 金额: --
• 依托单位: Leibniz-Zentrum Allgemeine Sprachwissenschaft (ZAS)
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2019
• 资助国家: 德国
• 起止时间: 2018-12-31 至 2022-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/426673330?language=en
• 关键词: Audio; visual; prosody; whispered; semi

Prosody plays a powerful role in speech. It helps listeners identify the boundaries of words and phrases, it highlights the most important pieces of information and it encodes emotions. While voiced speech prosody is an established research topic in phonetics and phonology, the prosody of whispered speech has received little attention so far. What is more, prosody in semi-whispered speech has not yet been investigated. The main goal of the present project is to provide more insight into the prosody of whispered and semi-whispered speech.The challenging nature of this goal lies in the fact that fundamental frequency, one of the main correlates of prosodic variation, is entirely absent in whispered speech. Thus, we propose an analysis of prosody through other acoustic parameters such as intensity, duration, and spectral properties of vowels and consonants. In our analysis we pay special attention to the still understudied interaction of prosody with consonants.In addition, we aim to extend the studied parameter space to the gestural dimension by taking advantage of the possibilities of motion tracking technology. The planned experiments include conditions that contrast communication in the acoustic-only with the joint acoustic-and-visual channel. We are interested in the extent to which acoustic cues may differ in these conditions, as well as in how and if visual prosody augments or supplants auditory cues in (semi-)whispered speech.Finally, the proposed investigation paves the way for a larger enterprise whose ultimate goal is to design algorithms which will convert semi-whispered and whispered speech to voiced speech. In the present project we concentrate on semi-whispered speech and aim to implement the results of the acoustic study in a semi-whisper-to-voiced conversion system, to be tested on speech that is not fully voiced (ILSE Corpus). Ultimately such a system will also help relieve the daily challenges of the ageing and clinical populations suffering from ailments such as throat cancers or neuro-degenerative diseases including Parkinson’s.

韵律在言语中起着重要的作用。它可以帮助听众识别单词和短语的界限，突出最重要的信息，并对情绪进行编码。浊音韵律是语音学和音韵学领域的一个既定研究课题，而对轻声语音韵律的研究迄今为止还很少受到关注。更重要的是，半耳语语音的韵律尚未被研究过。本项目的主要目标是为耳语和半耳语语音的韵律提供更多的见解。这一目标的挑战性在于，作为韵律变化的主要相关因素之一，基本频率在耳语中是完全不存在的。因此，我们建议通过其他声学参数，如元音和辅音的强度、持续时间和频谱特性来分析韵律。在我们的分析中，我们特别关注尚待研究的韵律与辅音的相互作用。此外，我们的目标是利用运动跟踪技术的可能性，将研究的参数空间扩展到手势维度。计划中的实验包括将声学通信与声视联合通道的通信进行对比的条件。我们感兴趣的是声音线索在这些情况下的差异程度，以及视觉韵律如何以及是否在（半）低声说话中增强或取代听觉线索。最后，所提出的研究为更大的企业铺平了道路，其最终目标是设计将半耳语和耳语语音转换为浊音语音的算法。在本项目中，我们专注于半耳语语音，并旨在将声学研究结果应用于半耳语到发声的转换系统中，并在不完全发声的语音（ILSE语料库）上进行测试。最终，这样一个系统还将有助于缓解老年人和临床人群的日常挑战，这些人患有喉癌或包括帕金森病在内的神经退行性疾病。