Collaborative Research: Deep learning speech recognition for documenting Seneca, a Native American language, and other acutely under-resourced languages

16世纪30年代，当探险家雅克·卡蒂埃沿着圣劳伦斯河航行时，第一次接触到了六国联盟或豪德诺索尼人的易洛魁语。字典和语法是存在的，但每种语言的文档的基本元素还应该包括不同类型的注释文本。正如美国国会于1990年通过的《美洲原住民语言法》所承认的那样，北美原住民使用的语言具有独特的地位和重要性。这个项目将把塞尼卡印第安人国家的成员与一个语言学和计算机研究人员小组聚集在一起，记录说塞尼卡语的老年人，塞尼卡语是一种特别濒危的易洛魁语。该团队将开发软件，使用自动语音识别（ASR）准确有效地转录这些录音，ASR是Siri或Alexa等数字个人助理背后的技术。塞涅卡有一个极其复杂的词结构，称为多合成，其中一个词相当于一个从句或句子。这样的语言挑战了ASR系统，ASR系统通常被设计为在受限的词汇表上识别单词。该项目将通过开发用于生成合成文本数据的新方法来推进科学知识，以增加建模这种复杂性所需的现有书面资源。更广泛的影响包括提供新记录的材料，用于语言振兴和科学调查。该项目将为Seneca Nation的本科生，研究生和年轻人提供宝贵的STEM经验，并扩大美洲原住民在语言和计算科学方面的参与，包括支持Seneca计算机科学博士生。其他致力于记录和分析低资源语言的人也可以使用所开发的计算工具和方法，其中许多语言在对国家安全至关重要的地区使用。塞涅卡语中的自发言语包含长而复杂的单词，但也有许多对理解话语至关重要的短粒子。对塞涅卡口语进行切分和注释的关键是出现在较长话语中的韵律模式，包括韵律和音调成分。大多数ASR框架将受到多合成形态系统倾向于产生的大词汇量的挑战。此外，ASR系统通常不对高级韵律信息进行建模。Seneca几乎没有来自自发语音的可用文本数据，这是建立ASR中使用的预测语言模型所必需的，对Seneca学习者来说是非常宝贵的。增强可用的文本数据将需要新的技术来生成合成但合理的文本，特别关注神经序列到序列模型。神经网络的能力，以模拟长距离和层次关系也将被利用，以捕捉话语级的韵律模式所需的准确分割的自发语音在塞涅卡。该项目汇集了一系列专业知识，并让塞尼卡社区成员、语言的主要利益攸关方参与进来，从而将传统的语言学方法和计算方法联系起来。通过跨学科合作转录和注释的每一个新的Seneca录音都将支持Seneca语言的复兴，并有助于促进低资源语言技术的发展。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The Iroquoian languages of the Six Nations Confederacy, or the Haudenosaunee people, were first encountered when the explorer Jacques Cartier sailed up the St. Lawrence River in the 1530s. Dictionaries and grammars exist, but the basic elements of documentation for every language should also include annotated texts of diverse genres. As currently recognized in The Native American Languages Act, passed by the U.S. Congress in 1990, languages spoken by the indigenous peoples of North America have a unique status and importance. This project will bring together members of the Seneca Nation of Indians with a team of linguistics and computing researchers to record elders speaking Seneca, an Iroquoian language that is particularly endangered. The team will develop software to accurately and efficiently transcribe these recordings using automatic speech recognition (ASR), the technology behind digital personal assistants like Siri or Alexa. Seneca has an exceedingly complex word structure, known as polysynthesis, in which a word is equivalent to a clause or sentence. Such languages challenge ASR systems, which are generally designed to recognize words over a constrained vocabulary. This project will advance scientific knowledge by developing novel methods for generating synthetic text data to augment the existing written resources required to model this complexity. Broader impacts include the availability of the newly documented materials for language revitalization and scientific investigation. The project will provide undergraduates, graduate students, and young adults from the Seneca Nation with valuable STEM experience and broadening participation of Native Americans in the language and computing sciences, including supporting a Seneca doctoral student in computer science. The computational tools and methodologies developed will be accessible to others who are working to document and analyze low-resource languages, many spoken in regions of critical importance for national security.Spontaneous speech in Seneca contains long, complex words but also many short particles that are essential to understanding the discourse. Crucial for segmenting and annotating spoken Seneca are the prosodic patterns that occur in longer utterances, involving both metrical and tonal components. Most ASR frameworks would be challenged by the large vocabulary size that a polysynthetic morphological system tends to yield. In addition, ASR systems do not typically model high-level prosodic information. Seneca has little available text data derived from spontaneous speech, which is needed to build the predictive language models used in ASR and is invaluable to Seneca learners. Augmenting the available text data will require novel techniques for generating synthetic but plausible text, with a particular focus on neural sequence-to-sequence models. The ability of neural nets to model long-distance and hierarchical relationships will also be exploited to capture utterance-level prosodic patterns required for accurate segmentation of spontaneous speech in Seneca. By bringing together a range of expertise and by involving Seneca community members, key stakeholders in the language, the project bridges traditional linguistic methodology and computational approaches. Each new Seneca recording that is transcribed and annotated through this collaboration across disciplines will support the revitalization of the Seneca language and help to further the state of the art in low-resource language technology.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.