Identifying mechanisms of multisensory memory using virtual reality and fMRI

项目摘要 对物体和事件形成和检索记忆是人类经验的基础。互补 内侧颞叶（MTL）内的机制支持对物体的记忆的形成和检索， 这样，嗅周皮层支持对单个项目特征的记忆，海马支持 在最初遇到的背景下回忆物体。虽然大多数关于物体的研究 记忆已经被单独使用视觉刺激进行，一小部分研究表明，视觉物体 与它们的特征声音一起沿着编码（例如，一只狗和一只树皮），后来被更好地记住， 仅在视觉上编码的对象，或具有无意义或非特征性声音的对象。尽管有这些证据， 多模态刺激在自然环境中的普遍存在，我们对多感官刺激程度的了解， 对记忆及其潜在机制的影响非常有限。我最近的行为研究表明， 多感官编码的好处是基于对这些物体所在环境的记忆的改善， 这表明，多感觉处理独特地参与了绑定这些对象的机制， 到周围的信息。然而，编码上下文的特征是什么仍然不清楚 更好地检索，以及什么神经机制是由多感觉处理调制，以允许这一点 效益拟议的研究通过收集关键证据来解决这一知识差距， 确定受多感官影响的项目和周围事件的记忆方面 处理，以及大脑活动模式和区域，参与检索这些信息。我们 假设编码时的多感觉处理通过增加 对象和上下文将被绑定到情景记忆中，以支持以后对该对象的识别， 而不是通过增强对单个对象本身的记忆。此外，我们预测， 在提取多模态对象期间海马体内的神经元也将携带关于环境的信息 这些物品的编码为了严格测试多感官处理的记忆类型 影响，有必要评估嵌入丰富， 自然主义的时空背景为此，我们开发了一种沉浸式的自然主义编码任务， 在虚拟现实（VR）环境中，包含受控但动画的视觉和视听对象。 这种新的方法将使我们能够评估多感觉处理是否会影响个体的记忆。 如果这些对象更容易与周围环境绑定以支持对事件的记忆， 目标1（Aim 1）。此外，fMRI数据的分析将用于确定MTL的哪些区域， 大脑皮层特别参与对在上下文中看到的多感觉物体的记忆（目标2）。总体 这个项目的目标是推进我们的知识，情节记忆是如何形成的自然过程中， 这项工作将有助于我们解释和预测现实世界行为的能力。

Project Summary Forming and retrieving memories for objects and events is fundamental to human experience. Complementary mechanisms within the medial temporal lobe (MTL) support the formation and retrieval of memories for objects, such that the perirhinal cortex supports memory for features of individual items, and the hippocampus supports recollection of the object within the context it was originally encountered. Although most research on object memory has been conducted using visual stimuli alone, a small body of research has shown that visual objects encoded along with their characteristic sound (e.g., a dog and a bark) are better remembered later on than objects encoded only visually, or with a meaningless or non-characteristic sound. Despite this evidence and the ubiquity of multimodal stimuli within natural environments, our knowledge of the extent of multisensory influences on memory and its underlying mechanisms is very limited. My recent behavioral work showed that the benefits of multisensory encoding are based on improved recollection of the context in which those objects were encoded, suggesting that multisensory processing uniquely engages mechanisms that bind these objects to surrounding information at encoding. However, it remains unclear what features of the encoding context are better retrieved, and what neural mechanisms are modulated by multisensory processing to allow for this benefit. The proposed research addresses this gap in knowledge by collecting the critical evidence to determine the facets of memory for items and surrounding events that are influenced by multisensory processing, and the brain activity patterns and regions that are involved in retrieving such information. We hypothesize that multisensory processing at encoding improves object memory by increasing the likelihood that the object and context will be bound into an episodic memory to support later recognition of that object, rather than by enhancing memory for the individual object itself. Further, we predict that patterns of activation within the hippocampus during retrieval of multimodal objects will also carry information about the environment in which these items were encoded. To rigorously test the type of memory that multisensory processing impacts, there is a need to assess memory for unimodal and multimodal objects that are embedded within rich, naturalistic spatiotemporal context. To this end, we have developed an immersive, naturalistic encoding task within virtual reality (VR) environments, which contain controlled but animated visual and audiovisual objects. This novel approach will allow us to assess whether multisensory processing influences memory for individual objects alone or if these objects are more readily bound to their surroundings to support memory for events within context (Aim 1). Further, analyses of fMRI data will be used to determine which regions of the MTL and cortex are specifically involved in memories of multisensory objects seen in context (Aim 2). The overarching goal of this project is to advance our knowledge of how episodic memories are formed during naturalistic experiences and this work will contribute to our ability to explain and predict real-world behaviors.