Advancing Neural Network Models of Language Processing

冲击力。了解语言处理对每个加拿大人的福祉都至关重要。加拿大各省教育部在与语言相关的特别项目上花费了45亿美元，包括为新来者提供语言培训，这证明了这个话题的重要性。这也体现在脑启发模型为人工智能(AI)带来的重大产业机会上，例如微软自然语言AI产生的100亿美元收入。因此，大脑语言系统的改进理论将使语言教学、补救和人工智能计划的一系列创新方法成为可能。我的研究重点是通过逐步建立和测试由模拟神经元组组成的语言神经网络模型来提高我们对文字处理的神经基础的理解。我开发的模型通过结合系统和细胞神经科学的额外原理，改进了经典的、极具影响力的神经网络模型。我的长期目标和相关的短期目标将在我之前对语言的以下关键方面的研究的基础上开辟新的天地：目标1：建立神经网络中语言的学习、表示和泛化的统一理论：这项工作将研究神经网络如何将语言中的规则概括为新学习的单词(像gint这样的新词可能与mint和lint的常规发音押韵，而不是pint的特殊发音)。这项研究将告诉我们，在第一语言和第二语言习得过程中，如何通过对单词样本的教学来实现对整个语言的广泛概括。目标2：建立一个在多种语言中有效的语言处理模型：到目前为止，大多数模型研究主要集中在英语上。这项研究将研究神经网络如何适应不同语言的特性来解释关键的跨语言差异(例如，希伯来语和英语读者对字母位置的差异敏感性)。反过来，这将告知神经网络是如何由一系列语言的特性塑造的，这可以改善许多语言的语言教学和补救。目标3：建立歧义字词加工与任务绩效的模型。与这一目标相关的模拟和实证工作将确定与语义模糊的单词(例如，银行在某些上下文中指河流，在另一些上下文中指金融机构)相关的一系列影响是由于意义选择的时间进程，还是决策系统如何利用语言系统。这些研究将为理解障碍的理论提供信息，并有助于与大脑启发的人工智能系统如何识别单词含义相关的工业应用。

Impact. Understanding language processing is critical to the well-being of every Canadian. The importance of this topic is exemplified by the $4.5 billion that provincial education ministries across Canada spend on special language-related programs, including language training for newcomers. It is also seen in the major industrial opportunities brain-inspired models have for artificial intelligence (AI), such as the $10 billion in revenue generated by Microsoft's natural language AI. An improved theory of the brain's language system will therefore enable a range of innovative approaches to language instruction, remediation, and AI.Plan. My research focuses on advancing our understanding of the neural basis of word processing by incrementally building and testing neural network models of language made up of groups of simulated neurons. The models that I develop improve upon classic, highly influential neural network models by incorporating additional principles from systems and cellular neuroscience. My long term objectives and associated short-term aims will break new ground by building upon my prior research on the following key aspects of language: Objective 1: Develop a unified theory of learning, representation, and generalization of language in neural networks: This work will examine how neural networks can generalize regularities in a language to newly learned words (a new word like gint probably rhymes with the regular pronunciation in mint and lint, not the exceptional pronuncation of pint). This research will inform how teaching a sample of words can yield widespread generalization to an entire language during first and second language acquisition. Objective 2: Create a model of language processing validated in multiple languages: Most modelling research to date has focused primarily on English. This research will study how neural networks adapt to the properties of different languages to explain key cross-linguistic differences (e.g., differential sensitivity to letter position in Hebrew vs. English readers). In turn, this will inform how neural networks are shaped by the properties of a range of languages, which can improve language instruction and remediation in many languages. Objective 3: Develop a model of ambiguous word processing and task performance. Simulations and empirical work related to this objective will determine whether a range of effects associated with semantically ambiguous words (e.g., bank, refers to a river in some contexts and a financial institute in others) are due to the time-course of meaning selection, or how the decision system taps into the language system. These investigations will inform theories of comprehension impairments, and contribute to industrial applications related to how brain-inspired AI systems identify a word's meaning.