EAGER: Exploring Cognitively Plausible Computational Models for Processing Human Language

尽管最近设计用于处理人类语言的人工智能技术取得了成功，但大多数当代解决方案都是为处理非常具体的语言处理任务而设计的。因此，对于大多数当前的计算方法来说，人类层面的语言理解仍然遥不可及，特别是在涉及到保留新信息和对积累的知识进行推理的情况下。这一探索性项目的目标是开发更符合认知现实的计算模型，该模型可以模拟人类语言处理的一些已知特性，因此更健壮，更适合作为一般语言理解系统，具有类似人类的学习，涉及随着时间的推移获取和更新知识。尽管大多数当代自然语言处理的深度学习方法侧重于特定于任务的端到端模型，但该项目优先考虑与当前数据一致的通才结构，这些结构将与当前数据保持一致，如语义启动、概念系统形成和使用中的分组和组块效应，以及长和短时间记忆对知识存储和检索的影响。在这个项目中，设计了新的神经网络结构，对这些属性的子集进行建模。通过加强大脑中的突触连接来实现学习和记忆的过程将被一组具有双向连接的代表性单位(r单位)模拟，模拟信息处理过程中新皮质小区域之间的相互作用。存储器激活状态模型根据应用于存储器存储器的卷积滤波器来表示r单元之间的连接。启动效应将由通过一系列去卷积操作产生的预激活模式来模拟。基于速率的连通性网络模型结合了基于每个节点的强化学习和应用于时变系统的Hebbian学习的形式，其中每个r单元计算其输出的变化率，允许节点激活在时间上徘徊；它使用离散的全局奖励信号进行训练。这个项目的目标是通过开发最初的概念验证原型，证明它们能够汇聚到简单的学习任务中，并将它们应用到语言建模任务中，以确保能够学习实际有用的表示法，从而确定所建议的体系结构的可行性。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Despite the recent successes of artificial intelligence techniques designed to process human language, most contemporary solutions are designed to handle very specific language processing tasks. As a result, human-level language understanding is still out of reach for most current computational approaches, especially when retaining new information and reasoning over the accumulated knowledge is involved. This exploratory project advances the goal of developing more cognitively realistic computational models that can mimic some of the known properties of human language processing, and as a result, be more robust and better suited as general systems for language understanding, with human-like learning which involves obtaining and updating knowledge over time.While most contemporary deep learning approaches in natural language processing focus on task-specific end-to-end models, this project prioritizes generalist architectures that would be consistent with the current data on semantic priming, grouping and chunking effects in the formation and use of conceptual systems, and the effects of long- and short-term memory on the storage and retrieval of knowledge. In this project, novel neural network architectures are planned that model a subset of these properties. The processes that enable learning and memory via strengthening of synaptic connections in the brain will be emulated by a set of representational units (r-units) with bidirectional connections, modeling the interaction between small regions of neocortex during information processing. Memory Store Activation State Model represents the connections between r-units in terms of convolutional filters applied to the memory store. The priming effects will be modeled by a pre-activation pattern produced via a sequence of deconvolutional operation. Rate-Based Connectivity Network model combines reinforcement learning on per-node basis with a form of Hebbian learning applied to a time-varying system where each r-unit calculates rate of change of its output, allowing node activations to linger through time; it is trained with a discrete global reward signal. The goal of this project is to establish the feasibility of the proposed architectures by developing the initial proof-of-concept prototypes, demonstrating that they are able to converge on simple learning tasks, and applying them to the task of language modeling to ensure that a practically useful representation can be learned.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.