Advancing Neural Network Models of Language Processing

冲击 了解语言处理对每个加拿大人的福祉至关重要。这一主题的重要性体现在加拿大各省教育部在特殊语言相关项目上花费的45亿加元，包括对新移民的语言培训。这也体现在大脑启发模型为人工智能（AI）带来的主要工业机会中，例如微软的自然语言AI创造了100亿美元的收入。 因此，大脑语言系统的改进理论将为语言教学、补救和人工智能提供一系列创新方法。计划我的研究重点是通过逐步构建和测试由模拟神经元组组成的语言神经网络模型来提高我们对文字处理神经基础的理解。我开发的模型通过结合系统和细胞神经科学的其他原理，改进了经典的、具有高度影响力的神经网络模型。 我的长期目标和相关的短期目标将通过建立在我之前对语言的以下关键方面的研究来开辟新天地：** 目标1：开发神经网络中语言学习，表示和泛化的统一理论：这项工作将研究神经网络如何将一种语言中的单词概括为新学习的单词（像gint这样的新词可能与mint和lint的常规发音押韵，而不是pint的特殊发音）。这项研究将告知如何教一个样本的话可以产生广泛的推广到整个语言在第一和第二语言习得。 ** 目标2：创建一个在多种语言中验证的语言处理模型：迄今为止，大多数建模研究主要集中在英语上。 这项研究将研究神经网络如何适应不同语言的特性，以解释关键的跨语言差异（例如，希伯来语读者与英语读者对字母位置的不同敏感性）。 反过来，这将告知神经网络如何由一系列语言的属性塑造，这可以改善许多语言的语言教学和补救。** 目标3：建立模糊文字处理和任务绩效的模型。 与这一目标相关的模拟和实证工作将确定与语义模糊的单词（例如，银行，在某些语境中指河流，在另一些语境中指金融机构）是由于意义选择的时间过程，或者决策系统如何进入语言系统。 这些研究将为理解障碍的理论提供信息，并有助于与大脑启发的人工智能系统如何识别单词含义相关的工业应用。 *****

Impact. Understanding language processing is critical to the well-being of every Canadian. The importance of this topic is exemplified by the $4.5 billion that provincial education ministries across Canada spend on special language-related programs, including language training for newcomers. It is also seen in the major industrial opportunities brain-inspired models have for artificial intelligence (AI), such as the $10 billion in revenue generated by Microsoft's natural language AI. An improved theory of the brain's language system will therefore enable a range of innovative approaches to language instruction, remediation, and AI.******Plan. My research focuses on advancing our understanding of the neural basis of word processing by incrementally building and testing neural network models of language made up of groups of simulated neurons. The models that I develop improve upon classic, highly influential neural network models by incorporating additional principles from systems and cellular neuroscience. My long term objectives and associated short-term aims will break new ground by building upon my prior research on the following key aspects of language: ******Objective 1: Develop a unified theory of learning, representation, and generalization of language in neural networks: This work will examine how neural networks can generalize regularities in a language to newly learned words (a new word like gint probably rhymes with the regular pronunciation in mint and lint, not the exceptional pronuncation of pint). This research will inform how teaching a sample of words can yield widespread generalization to an entire language during first and second language acquisition. ******Objective 2: Create a model of language processing validated in multiple languages: Most modelling research to date has focused primarily on English. This research will study how neural networks adapt to the properties of different languages to explain key cross-linguistic differences (e.g., differential sensitivity to letter position in Hebrew vs. English readers). In turn, this will inform how neural networks are shaped by the properties of a range of languages, which can improve language instruction and remediation in many languages. ******Objective 3: Develop a model of ambiguous word processing and task performance. Simulations and empirical work related to this objective will determine whether a range of effects associated with semantically ambiguous words (e.g., bank, refers to a river in some contexts and a financial institute in others) are due to the time-course of meaning selection, or how the decision system taps into the language system. These investigations will inform theories of comprehension impairments, and contribute to industrial applications related to how brain-inspired AI systems identify a word's meaning. *****