Three dimensions of sentence prosody

句子韵律的三个维度

• 批准号: RGPIN-2018-06153
• 负责人: Wagner, Michael
• 金额: $ 2.91万
• 依托单位: McGill University
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2019
• 资助国家: 加拿大
• 起止时间: 2019-01-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=688126
• 关键词: Three; dimensions; sentence; prosody

Speech synthesis has come a long way. Today's computers can produce words and sentences highly accurately, but if a computer reads out a novel text, the result is still often hard to listen to. A major reason is our lack of understanding of sentence prosody. Sentence prosody can be characterized as all those acoustic properties of an utterance that are not a function of the words it contains, but rather are due to other factors, and includes three separate dimensions: Intonation (e.g., falling intonation on assertions vs. rising intonation on questions), Phrasing (the way the words in a sentence group together), and Prominence (for example highly predictable or discourse given words are often reduced, while contrastive information is boosted). These three dimensions of sentence prosody (intonation, phrasing, prominence) are partly determined by the phonological representations of a particular language (e.g., declarative intonation in French sounds different than in English) and partly a direct reflex of on-line processing during speaking (e.g., complex constituents are often preceded by phrasing junctures). Once we understand the representational questions and can disentangle the three dimensions, sentence prosody can be used as a window into cognitive mechanisms underlying speech production. This proposal brings together several strands of research that my lab has contributed to in the past.******The first step to a better understanding of sentence prosody is to disentangle the three mentioned dimensions in the signal. The syntactic and semantic functions (type of speech act, grouping, contrast) are orthogonal to each other, but whether their prosodic correlates (tune, phrasing, prominence) are remains controversial. Most current models of sentence prosody assume interactions or even conflate the dimensions. However, the reality of the assumed interactions between the dimensions has not been systematically established. An initial study, conducted for this proposal in my lab (Wagner & McAuliffe 2017, Interspeech), systematically crossed the three functional dimensions (speech act, syntax, contrast) in a factorial speech production experiment. The results show that we can in fact recover the three `dimensions' of prosody quite reliably from speakers' utterances, once we factor out the contribution of the other dimensions, as well as speaker differences. While we find limited interactions, they are not actually the ones that current models predict. This project consists of a series of production and perception experiments designed to establish the true interactions, and develops a more appropriate representational model. We make use of novel tools (e.g, The Montreal Forced Aligner, developed in my lab, Mcauliffe et al. 2017, Interspeech) and statistical modelling, to develop models that can disentangle the dimensions, just as humans do when processing speech.**

语音合成已经走了很长的路。今天的计算机可以非常准确地产生单词和句子，但如果计算机读出一段小说文本，结果仍然很难听。一个主要原因是我们对句子韵律的认识不足。 句子韵律可以被表征为话语的所有那些声学特性，这些声学特性不是它所包含的单词的函数，而是由于其他因素，并且包括三个单独的维度：语调（例如，断言上的语调下降与问题上的语调上升），短语（句子中的单词组合在一起的方式）和突出（例如，高度可预测或话语给定的单词通常会减少，而对比信息会增加）。句子韵律的这三个维度（语调、措辞、突出）部分地由特定语言的语音表征决定（例如，法语中的声明性语调听起来与英语中的不同），并且部分地是说话期间在线处理的直接反射（例如，复合成分之前通常有短语连接）。一旦我们理解了表征性问题，并能够理清这三个维度，句子韵律就可以作为了解言语产生背后的认知机制的一个窗口。这份提案汇集了我的实验室过去所做的几项研究。要更好地理解句子韵律，第一步是要理清信号中提到的三个维度。句法功能和语义功能（言语行为类型、组合、对比）是相互正交的，但它们之间的韵律关联（声调、措辞、突显）是否存在争议。目前大多数句子韵律模型假设相互作用，甚至混淆了维度。然而，这些层面之间假定的相互作用的现实尚未得到系统的确立。在我的实验室里，针对这一提议进行了一项初步研究（瓦格纳和麦考利夫，2017年，Interspeech），在析因言语产生实验中系统地跨越了三个功能维度（言语行为，句法，对比）。结果表明，我们实际上可以恢复的韵律的三个“维度”相当可靠地从扬声器的话语，一旦我们因素的其他方面的贡献，以及扬声器差异。虽然我们发现有限的相互作用，但它们实际上并不是当前模型预测的相互作用。该项目包括一系列的生产和感知实验，旨在建立真正的相互作用，并开发一个更合适的代表性模型。我们利用新的工具（例如，在我的实验室开发的蒙特利尔强制对齐器，Mcauliffe et al. 2017，Interspeech）和统计建模，开发可以解开维度的模型，就像人类处理语音一样。