Text, Neuroimaging, and Memory: Unified Models of Corpora and Cognition

PI将开发新的机器学习算法，以探索意义在大脑中是如何呈现的，以及含义呈现如何塑造人类记忆。目前的神经科学记忆理论认为，形成对特定事件的记忆需要将该事件的细节与该人当前的心理环境联系起来，即她当时正在考虑的其他一切。当试图记住事件时，这个人可以通过恢复记忆形成时存在的心理环境来访问存储的细节。这符合一种直觉，即忘记的细节(例如，房子钥匙放错地方的位置)可以通过精神上的“重新追踪步骤”来找回，即试图恢复最初事件发生时的心态。考虑到这些理论，这项工作的目标是开发机器学习算法，使其能够基于功能磁共振成像大脑数据和行为记忆数据跟踪“心理回溯步骤”的过程-所提出的算法将能够解码一个人在形成记忆时以及(后来)当她搜索这些记忆时的精神语境的状态。拟议中的工作使用了关于记忆和意义的两个基本概念：第一个想法是，心理语境由最近遇到的刺激的含义塑造。第二个想法是，大脑中概念之间的语义关系反映了自然语言中单词之间的统计关系。开发的算法将汇集来自三个来源的数据-执行记忆回忆任务的受试者的行为数据，受试者执行这些任务时收集的fMRI神经成像数据，以及大量文档-以发现可以同时描述所有三种类型信息的潜在意义空间。这个空间中的每一点都描述了一个心理环境。因此，拟议工作的核心是开发潜变量模型和算法，这些模型和算法可以从数据中推断当一个人存储和搜索记忆时，心理背景如何在意义空间中移动。拟议的工作将导致机器学习(基于多种不同数据类型推断隐藏变量的新算法)和神经科学(更精细的记忆搜索是如何在大脑中完成的理论)的根本性进步。此外，这项工作将促进诊断和修复记忆问题的新技术的发展，使人们能够跟踪经历记忆提取失败的人的情境恢复过程是如何出错的。

The PIs will develop new machine learning algorithms to explore how meaning is represented in the brain and how meaning representations shape human memory. Current neuroscientific theories of memory posit that forming a memory for a particular event involves associating the details of that event with the person's current mental context, i.e., everything else that she is thinking about at the time. When trying to remember the event, the person can access stored details by reinstating the mental context that was present when the memory was formed. This fits with the intuition that forgotten details (e.g., the location of misplaced house keys) can be retrieved by mentally "re-tracing steps", i.e., trying to reinstate the mindset that was present at the time of the original event. With these theories in mind, the goal of this work is to develop machine learning algorithms that make it possible to track, based on fMRI brain data and behavioral memory data, the process of "mentally re-tracing steps"---the proposed algorithms will be able to decode the state of a person's mental context as she forms memories and (later) as she searches for these memories.The proposed work uses two fundamental ideas about memory and meaning: The first idea is that mental context is shaped by the meanings of recently encountered stimuli. The second idea is that semantic relationships between concepts in the brain mirror statistical relationships between words in naturally occurring language. The developed algorithms will bring together data from three sources---behavioral data from subjects performing memory recall tasks, fMRI neuroimaging data collected while subjects performed these tasks, and large collections of documents---to discover a latent meaning space that can simultaneously describe all three types of information. Each point in this space describes a mental context. Thus the core of the proposed work is to develop latent variable models and algorithms that can infer from data how the mental context moves through meaning space as a person stores and searches for memories.The proposed work will lead to fundamental advances in machine learning (new algorithms for inferring hidden variables based on multiple, heterogeneous data types) and neuroscience (more refined theories of how memory search is accomplished in the brain). Furthermore, this work will catalyze the development of new technologies for diagnosing and remediating memory problems, by making it possible to track how the contextual reinstatement process is going awry in people experiencing memory retrieval failure.