EAGER: Identifying and Producing Code-Switching in Languages from Spoken, Lexical and Socio-linguistic Features

这个探索性研究的早期赠款调查了我们世界上大量讲多种语言的人之间的对话，他们经常在这些语言之间来回切换，称为“代码转换”。对于语音对话系统和语音助理来说，重要的是不仅能够识别何时、为什么以及产生什么效果的代码切换发生，而且能够正确地解释所说的内容，并且能够在与这样的用户交互时生成类似的代码切换响应。近年来语音技术的进步导致了Siri、Google Assistant和Alexa等语音助手的广泛使用。它们可以极大地改善英语、法语、德语、广东话、普通话和西班牙语等语言的语音信息访问。然而，这种访问仅限于单语语音，对于许多多语言使用者来说，这不是最自然的语音产生形式。因此，语码转换的语音很少被正确理解，并且永远无法在辅助反应中产生。为这些说话者实现自然和舒适的交流的一个主要障碍是缺乏语音技术，这种语音技术不仅可以理解代码切换输入，而且还可以产生类似于人类的输出。这个项目通过研究口语和书面语码转换如何与语言交际的其他方面相互作用来解决这些问题。本文将探讨语码转换研究中尚未涉及的问题，包括：（1）说话者是否会在发音和其他语码转换策略上产生相似性;（2）语码转换与移情之间是否存在可量化的关系，移情是说话者表达理解他人的问题并希望帮助解决这些问题的意图;（3）命名实体（如姓名或地理位置）的出现是否会引发语码转换;（4）语码转换中最常出现的对话行为，如提问、陈述或反向通道;（5）说话者在语码转换时如何产生语调轮廓（通过选择他们的语调生产，以配合他们正在生产的语言或通过不同的两种？）统计和机器学习技术都将被用来解决这些问题的背景下，口语和词汇特征标记的代码转换语音标准美国英语，西班牙语，汉语普通话，和印地语。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This Early Grant for Exploratory Research investigates conversations between the vast number of persons in our world who speak multiple languages and who frequently switch back and forth between those languages in what is called “code-switching”. It is important for speech dialogue systems and voice assistants to not only be able to identify when, why, and to what effect code-switching occurs, but also to correctly interpret what is said and to be able to generate similarly code-switched responses when interacting with such users. Advances in speech technology in recent years have resulted in widespread use of voice assistants such as Siri, Google Assistant and Alexa. They enable vast improvement in information access by voice for languages such as English, French, German, Cantonese, Mandarin, and Spanish. However, such access is limited to monolingual speech, which for many multilingual speakers is not the most natural form of speech production. Thus, code-switched speech is rarely understood correctly and is never able to be produced in assistant responses. A major barrier to enabling naturalistic and comfortable communication for these speakers is the lack of speech technology that can not only understand code-switched input but also produce similar human-like output. This project addresses these issues by examining how spoken and written code-switching interacts with other aspects of language communication. It will explore research questions not yet studied in code-switching research including (1) whether speakers entrain, speak more similarly, on pronunciation and other strategies of code-switching in speech; (2) whether there is a quantifiable relationship between code-switching and empathy in speech, where empathy is a speaker's intention to convey that they understand another's problems and want to help address them; (3) whether the presence of named entities, such as names or geographical locations, primes code-switching; (4) which dialogue acts, such as questions or statements or backchannels, tend to be produced most often in code-switched speech; and (5) how speakers produce intonational contours when they code-switch (via choosing their intonation production to match either of the languages they are producing or by being different from both?) Statistical and machine-learning techniques will both be used to address these questions in the context of spoken and lexical-feature-tagged code-switched speech in Standard American English, Spanish, Mandarin Chinese, and Hindi. By identifying new aspects of code-switching, the project will seed further exploration of this phenomenon by the research community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.