Doctoral Dissertation Research: The integration of visual and auditory information in tone perception

不同的语言使用音调来传达意思的方式各不相同。在声调语言中，音高是用来区分词义的。例如，在普通话中，音节ma用稳定的高调说的意思是妈妈，但用降调说的意思是责骂。声调语言占世界6000种语言的60%-70%；其中约20%的声调听起来非常相似，非母语者很难区分。由于全球化、国际商务、移民和跨文化交流，世界各地越来越多的人正在学习声调语言。截至2013年，仅在美国学习中文的大学生就超过了6.1万人，到2050年，这个数字很可能会超过13.4万人。随着美国经济日益国际化，对外语熟练程度的需求越来越强烈，越南语和泰语等许多不太常用的声调语言吸引了大量学习者。然而，学习声调语言是具有挑战性的。研究表明，一般来说，人们在辨别和发音一门外语的声调方面存在困难，这阻碍了跨文化交流，因为发音错误会导致交流失误。因此，更好地学习声调语言将有助于美国商界与中国和其他使用声调语言的地区进行贸易。这个项目的重点是学习粤语声调的过程，广东话的声调对于非母语者来说很难区分。具体地说，这个项目旨在调查非母语人士在学习辨别和产生粤语声调时，是否可以从提供声标(一组显示音高轮廓的符号)中受益。这项研究将在3个地区进行：美国、泰国和香港，这些地方很容易找到以美国英语、普通话、泰语和广东话为母语的人。在美国，以美国英语为母语的人和以普通话为母语的人将参加一项以听力为基础的声调学习实验，学习在有或没有声标的情况下听和产生粤语声调的对比。虽然广东话和普通话都是汉语的变体，但它们的声调系统不同。普通话有四声(高、中、低、降)，而广东话有六声(高、中、低、高、低)。研究表明，说普通话的人在学习粤语声调方面存在困难。同样的实验也将在泰国进行，实验对象是以泰语为母语的人。以香港粤语为母语的人士将参与实验，以提供一个比较案例。调查人员会评估加入声调标记是否可以大大改善非母语人士辨别和产生粤语声调的能力。研究人员还将评估来自不同语言背景的人是否表现出不同的学习模式，以及他们是否能获得与母语相似的结果。这项研究的发现将为外语学习者在非母语声音学习过程中如何整合视觉和听觉信息提供深入的见解。研究结果还将显示人类在一般的语音处理中如何使用视觉和听觉信息。这项研究也将有可能改善声调语言的教学。如果声调符号被证明是有帮助的，它们可以很容易地被纳入教材，并作为一种教学方法，以获得更好的声调学习结果。

Languages vary in how they use the pitch of voice to convey meaning. In a tone language, pitch is used to differentiate word meanings. For example, in Mandarin Chinese, the syllable ma said with a steady high tone means 'mother', but said with a falling tone it means 'to scold'. Tone languages represent 60%-70% of the world's 6,000 languages; about 20% of them have tones that sound very similar to each other and are difficult for non-native speakers to distinguish. More and more people around the world are learning tone languages because of globalization, international business, migration, and cross-cultural communications. In the United States, the number of college students studying Chinese alone was over 61,000 as of 2013, and the number is likely to surpass 134,000 by 2050. As the economy of the United States has become increasingly international, the demand for foreign language proficiency is stronger and stronger, and many less commonly taught tone languages such as Vietnamese and Thai have attracted a large number of learners. However, learning a tone language is challenging. Research has shown that, in general, people have difficulties distinguishing and pronouncing tones in a foreign language and that this hinders cross-cultural communication, as miscommunication arises due to mispronunciation. Thus, better learning of tone languages would help the American business community in trade with China and other areas where tone languages are spoken.The focus of this project is the process for learning tones in Cantonese, a language with tones that are difficult for non-native speakers to distinguish. Specifically, this project aims to investigate whether non-native speakers can benefit from providing tone marks, a set of symbols to show pitch contour, in learning to distinguish and produce Cantonese tones. The research will be conducted in 3 locations: the United States, Thailand and Hong Kong, where native speakers of American English, Mandarin, Thai, and Cantonese are easily found. In the United States, native speakers of American English and native speakers of Mandarin will participate in a listening-based tone learning experiment to learn to hear and produce the contrasts of Cantonese tones with or without the help of tone marks. Although both Cantonese and Mandarin are varieties of Chinese, they have different tone systems. While Mandarin has four tones (high, mid rising, low-dipping and high falling), Cantonese has six (high, mid, low, high-rising, low-rising and low falling). Studies have shown that Mandarin speakers have difficulties learning Cantonese tones. The same experiment will be conducted in Thailand with native speakers of Thai. Native speakers of Cantonese in Hong Kong will participate in the experiment to provide a comparison case. The investigators will assess whether the addition of tone marks can result in substantial improvement in distinguishing and producing Cantonese tones by the non-native speakers. The investigators will also assess whether people from different language backgrounds exhibit different learning patterns and whether they can achieve native-like results. The findings from this research will provide insight into how foreign language learners integrate visual and auditory information during non-native sound learning. The results will also show how human beings use visual and auditory information in speech processing in general. This study will also have the potential to improve the teaching of tone languages. If tone marks are proven to be helpful, they can easily be included in teaching materials and used as a teaching method for better tone-learning outcomes.