Collaborative Research: Integrating shape, scaling, and alignment in a global approach to F0 events in intonation systems

协作研究：将形状、缩放和对齐整合到语调系统中 F0 事件的全局方法中

• 批准号: 1023954
• 负责人: Nanette Veilleux
• 金额: $ 3.36万
• 依托单位: Simmons University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-10-01 至 2015-01-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1023954&HistoricalAwards=false
• 关键词: Collaborative; Research; Integrating; shape; scaling

Human languages use pitch to convey meaning in a bewildering variety of ways. In all languages, pitch (as one aspect of speech prosody) can express attitude or emotion. In some languages, like English, pitch patterns, usually called intonation contours, also express distinctions such as that between a question and a statement. In languages like Mandarin Chinese, pitch patterns usually called tones go still further to signal differences between words that are otherwise identical. Despite significant advances in recent decades, a unified theoretical account of such linguistic phenomena remains elusive. What is missing is a common acoustic or articulatory vocabulary for expressing the relevant distinctions---a single measurable dimension within which spoken pitch contours (rises and falls) can be reliably distinguished regardless of the language under investigation. In recent work, the research team developed a new mathematical approach to tone and intonation, based on the notion of Tonal Center of Gravity. TCoG is a gestalt or global approach to tone perception and production that reconciles seemingly contradictory results from different strands of the experimental literature, moving toward a model that incorporates the best aspects of past theories, while avoiding their characteristic weaknesses. That earlier work has established that the TCoG approach accounts well for production and perception data involving two contrasting English intonation contours. This project aims to expand the empirical range of the approach in three crucial ways: First it extends the model to additional English intonation patterns. Second, it moves beyond English to look at other intonation languages (e.g., German), as well as so-called "tone languages" (e.g., Serbian). Lastly, whereas the previous work concentrated primarily on the timing of tonal events in speech, this project goes further, to investigate the interaction of tonal timing patterns with the scaling of tonal events in the pitch domain. The experimental work will be of two primary kinds: automatic classification of pitch contours recorded from native speakers in an experimental setting, and direct manipulation (through speech synthesis) of pitch contours in perception studies designed to determine which aspects of the acoustic signal have the greatest effect on listeners' judgments of utterance meaning.Given the central role of intonation patterns in speech communication, one major contribution of the Tonal Center of Gravity approach is its potential to transform methods for speech synthesis and speech understanding. Synthetic speech is typically described as repetitive, detached, and often unhelpfully neutral; listeners recognize that they are talking with a machine that ?doesn?t get it?. By providing a more detailed understanding of how intonational patterns help to convey a message, TCoG could be used to devise algorithms for the synthesis of more natural and appropriate-sounding speech. Likewise, for automatic understanding of the aspects of meaning that depend on intonational patterns, TCoG could allow an automatic system to detect levels of nuance beyond simply whether a word is emphasized or not, or whether an utterance is a statement or a question. A final application of this work could be in the development of software tools for second language learning, in which automated instruction and feedback on the subtleties of second language intonation patterns could help learners master important aspects of communication that are typically ignored in current approaches to language pedagogy.

人类语言使用音高以各种各样令人困惑的方式传达意思。在所有的语言中，音高（作为语音韵律的一个方面）都可以表达态度或情感。在某些语言中，如英语，音高模式，通常称为语调轮廓，也表达了诸如问题和陈述之间的区别。在像汉语普通话这样的语言中，通常被称为音调的音高模式更进一步地表明了在其他方面相同的单词之间的差异。尽管近几十年来取得了重大进展，但对这种语言现象的统一理论解释仍然难以捉摸。所缺少的是用于表达相关区别的常见声学或发音词汇--一个单一的可测量维度，在该维度内，无论所研究的语言如何，都可以可靠地区分口语音调轮廓（上升和下降）。福尔斯。在最近的工作中，研究小组开发了一种新的数学方法来研究音调和语调，基于音调重心的概念。TCoG是一种完形或整体方法来感知和生产音调，调和看似矛盾的结果从不同的实验文献，走向一个模型，结合了过去的理论最好的方面，同时避免他们的特点弱点。早期的工作已经确定，TCoG方法很好地解释了涉及两种对比英语语调轮廓的产生和感知数据。 本项目旨在从三个关键方面扩大该方法的实证范围：首先，它将该模型扩展到其他英语语调模式。其次，它超越了英语，看看其他语调语言（例如，德语），以及所谓的“声调语言”（例如，塞尔维亚语）。最后，而以前的工作主要集中在语音中的音调事件的时间，这个项目更进一步，调查音调的时间模式与音调事件的缩放在音高域的相互作用。实验工作将主要分为两类：在实验环境中，对从母语者记录的音高轮廓进行自动分类，并直接操纵在感知研究中，音调轮廓（通过语音合成）的研究旨在确定声学信号的哪些方面对听者对话语意义的判断有最大的影响。鉴于语调模式在语音交际中的核心作用，音调重心方法的一个主要贡献是其转换用于语音合成和语音理解的方法的潜力。合成语音通常被描述为重复的，分离的，往往无益的中立;听众认识到，他们正在与一台机器？有吗你不明白吗？通过提供对语调模式如何帮助传达信息的更详细的理解，TCoG可以用来设计合成更自然和适当的语音的算法。同样，为了自动理解依赖于语调模式的意义方面，TCoG可以允许自动系统检测细微差别的级别，而不仅仅是一个词是否被强调，或者一个话语是一个陈述还是一个问题。这项工作的最后一个应用可能是开发第二语言学习的软件工具，其中对第二语言语调模式的微妙之处的自动化教学和反馈可以帮助学习者掌握在当前语言教学方法中通常被忽视的重要方面。