Doctoral Dissertation Research: The integration of visual and auditory information in tone perception

语言在如何使用音高来传达意思方面有所不同。在声调语言中，音高是用来区分词义的。例如，在普通话中，音节ma用稳定的高音表示“母亲”，但用降调表示“母亲”。声调语言占世界6,000种语言的60%-70%;其中约20%的声调听起来非常相似，非母语人士很难区分。由于全球化、国际商务、移民和跨文化交流，世界上越来越多的人正在学习声调语言。在美国，截至2013年，仅学习中文的大学生人数就超过61，000人，到2050年，这一数字可能超过134，000人。随着美国经济日益国际化，对外语能力的需求越来越强烈，许多不太常见的声调语言，如越南语和泰语，吸引了大量的学习者。然而，学习声调语言是一项挑战。研究表明，一般来说，人们在外语中难以区分和发音，这阻碍了跨文化交流，因为错误的发音会导致误解。因此，更好地学习声调语言将有助于美国商界与中国和其他使用声调语言的地区进行贸易。本项目的重点是学习粤语声调的过程，粤语的声调对非母语人士来说很难区分。具体而言，本计划旨在研究非母语人士能否从提供声调标记（一组显示音高轮廓的符号）中获益，以学习辨别和发出粤语声调。该研究将在美国、泰国和香港三个地方进行，这些地方的母语是美国英语、普通话、泰语和粤语。在美国，母语是美国英语的人和母语是普通话的人将参加一个基于语音的声调学习实验，学习在有或没有声调标记的帮助下听到和产生粤语声调的对比。虽然粤语和普通话都是汉语的变体，但它们有不同的声调系统。普通话有四个声调（高、中升、低降和高降），广东话有六个声调（高、中、低、高升、低升和低降）。研究表明，说普通话的人学习广东话声调有困难。同样的实验将在泰国进行，母语为泰国语的人。以广东话为母语的香港人将参与实验，提供一个比较案例。研究人员会评估加入声调标记后，会否大大改善非母语人士分辨和发出广东话声调的能力。研究人员还将评估来自不同语言背景的人是否表现出不同的学习模式，以及他们是否能达到与母语相似的结果。 本研究的发现将有助于我们了解外语学习者在非母语语音学习中是如何整合视觉和听觉信息的。结果还将显示人类如何在语音处理中使用视觉和听觉信息。这项研究也将有可能改善声调语言的教学。如果声调标记被证明是有帮助的，它们可以很容易地被纳入教材，并作为一种教学方法，以获得更好的声调学习效果。

Languages vary in how they use the pitch of voice to convey meaning. In a tone language, pitch is used to differentiate word meanings. For example, in Mandarin Chinese, the syllable ma said with a steady high tone means 'mother', but said with a falling tone it means 'to scold'. Tone languages represent 60%-70% of the world's 6,000 languages; about 20% of them have tones that sound very similar to each other and are difficult for non-native speakers to distinguish. More and more people around the world are learning tone languages because of globalization, international business, migration, and cross-cultural communications. In the United States, the number of college students studying Chinese alone was over 61,000 as of 2013, and the number is likely to surpass 134,000 by 2050. As the economy of the United States has become increasingly international, the demand for foreign language proficiency is stronger and stronger, and many less commonly taught tone languages such as Vietnamese and Thai have attracted a large number of learners. However, learning a tone language is challenging. Research has shown that, in general, people have difficulties distinguishing and pronouncing tones in a foreign language and that this hinders cross-cultural communication, as miscommunication arises due to mispronunciation. Thus, better learning of tone languages would help the American business community in trade with China and other areas where tone languages are spoken.The focus of this project is the process for learning tones in Cantonese, a language with tones that are difficult for non-native speakers to distinguish. Specifically, this project aims to investigate whether non-native speakers can benefit from providing tone marks, a set of symbols to show pitch contour, in learning to distinguish and produce Cantonese tones. The research will be conducted in 3 locations: the United States, Thailand and Hong Kong, where native speakers of American English, Mandarin, Thai, and Cantonese are easily found. In the United States, native speakers of American English and native speakers of Mandarin will participate in a listening-based tone learning experiment to learn to hear and produce the contrasts of Cantonese tones with or without the help of tone marks. Although both Cantonese and Mandarin are varieties of Chinese, they have different tone systems. While Mandarin has four tones (high, mid rising, low-dipping and high falling), Cantonese has six (high, mid, low, high-rising, low-rising and low falling). Studies have shown that Mandarin speakers have difficulties learning Cantonese tones. The same experiment will be conducted in Thailand with native speakers of Thai. Native speakers of Cantonese in Hong Kong will participate in the experiment to provide a comparison case. The investigators will assess whether the addition of tone marks can result in substantial improvement in distinguishing and producing Cantonese tones by the non-native speakers. The investigators will also assess whether people from different language backgrounds exhibit different learning patterns and whether they can achieve native-like results. The findings from this research will provide insight into how foreign language learners integrate visual and auditory information during non-native sound learning. The results will also show how human beings use visual and auditory information in speech processing in general. This study will also have the potential to improve the teaching of tone languages. If tone marks are proven to be helpful, they can easily be included in teaching materials and used as a teaching method for better tone-learning outcomes.