Collaborative Research: Integrating shape, scaling, and alignment in a global approach to F0 events in intonation systems

人类语言利用音高以各种令人眼花缭乱的方式传达意思。在所有语言中，音高（作为言语韵律的一个方面）都可以表达态度或情感。在一些语言中，比如英语，音高模式，通常被称为语调轮廓，也表达了区别，比如疑问和陈述句之间的区别。在像普通话这样的语言中，通常被称为音调的音高模式进一步表明了其他相同单词之间的差异。尽管近几十年来取得了重大进展，但对这种语言现象的统一理论解释仍然难以捉摸。缺少的是一种共同的声学或发音词汇来表达相关的区别——一个单一的可测量的维度，在这个维度内，无论研究的语言是什么，都可以可靠地区分语音音高的轮廓（上升和下降）。在最近的工作中，研究小组基于音调重心的概念，开发了一种新的音调和语调的数学方法。TCoG是一种对音调感知和产生的格式塔或全球方法，它调和了来自不同实验文献的看似矛盾的结果，朝着一个融合了过去理论的最佳方面的模型前进，同时避免了它们的特征弱点。早期的工作已经证明，TCoG方法可以很好地解释涉及两种对比的英语语调轮廓的产生和感知数据。本项目旨在从三个关键方面扩展该方法的经验范围：首先，它将模型扩展到其他英语语调模式。其次，它超越英语，着眼于其他语调语言（如德语），以及所谓的“声调语言”（如塞尔维亚语）。最后，鉴于之前的工作主要集中在语音中音调事件的时间上，本项目进一步研究了音调时间模式与音调事件在音调域的缩放之间的相互作用。实验工作将主要分为两类：在实验环境中对母语者记录的音高轮廓进行自动分类，以及在感知研究中直接操纵（通过语音合成）音高轮廓，旨在确定声学信号的哪些方面对听者对话语意义的判断影响最大。鉴于语调模式在语音交流中的核心作用，音调重心方法的一个主要贡献是它有可能改变语音合成和语音理解的方法。合成语音通常被描述为重复的、超然的、往往毫无帮助的中立；听者意识到他们是在和一台机器交谈，而这台机器不知道他们在说什么。我明白了吗？通过提供对语调模式如何帮助传递信息的更详细的理解，TCoG可以用来设计算法来合成更自然、更合适的语音。同样，对于依赖于语调模式的意义方面的自动理解，TCoG可以允许一个自动系统检测细微差别的水平，而不仅仅是一个单词是否被强调，或者一个话语是陈述还是疑问。这项工作的最后一个应用可能是开发第二语言学习的软件工具，其中自动指导和反馈第二语言语调模式的微妙之处可以帮助学习者掌握交际的重要方面，这些方面在当前的语言教学方法中通常被忽视。

Human languages use pitch to convey meaning in a bewildering variety of ways. In all languages, pitch (as one aspect of speech prosody) can express attitude or emotion. In some languages, like English, pitch patterns, usually called intonation contours, also express distinctions such as that between a question and a statement. In languages like Mandarin Chinese, pitch patterns usually called tones go still further to signal differences between words that are otherwise identical. Despite significant advances in recent decades, a unified theoretical account of such linguistic phenomena remains elusive. What is missing is a common acoustic or articulatory vocabulary for expressing the relevant distinctions---a single measurable dimension within which spoken pitch contours (rises and falls) can be reliably distinguished regardless of the language under investigation. In recent work, the research team developed a new mathematical approach to tone and intonation, based on the notion of Tonal Center of Gravity. TCoG is a gestalt or global approach to tone perception and production that reconciles seemingly contradictory results from different strands of the experimental literature, moving toward a model that incorporates the best aspects of past theories, while avoiding their characteristic weaknesses. That earlier work has established that the TCoG approach accounts well for production and perception data involving two contrasting English intonation contours. This project aims to expand the empirical range of the approach in three crucial ways: First it extends the model to additional English intonation patterns. Second, it moves beyond English to look at other intonation languages (e.g., German), as well as so-called "tone languages" (e.g., Serbian). Lastly, whereas the previous work concentrated primarily on the timing of tonal events in speech, this project goes further, to investigate the interaction of tonal timing patterns with the scaling of tonal events in the pitch domain. The experimental work will be of two primary kinds: automatic classification of pitch contours recorded from native speakers in an experimental setting, and direct manipulation (through speech synthesis) of pitch contours in perception studies designed to determine which aspects of the acoustic signal have the greatest effect on listeners' judgments of utterance meaning.Given the central role of intonation patterns in speech communication, one major contribution of the Tonal Center of Gravity approach is its potential to transform methods for speech synthesis and speech understanding. Synthetic speech is typically described as repetitive, detached, and often unhelpfully neutral; listeners recognize that they are talking with a machine that ?doesn?t get it?. By providing a more detailed understanding of how intonational patterns help to convey a message, TCoG could be used to devise algorithms for the synthesis of more natural and appropriate-sounding speech. Likewise, for automatic understanding of the aspects of meaning that depend on intonational patterns, TCoG could allow an automatic system to detect levels of nuance beyond simply whether a word is emphasized or not, or whether an utterance is a statement or a question. A final application of this work could be in the development of software tools for second language learning, in which automated instruction and feedback on the subtleties of second language intonation patterns could help learners master important aspects of communication that are typically ignored in current approaches to language pedagogy.