Identifying mechanisms of multisensory memory using virtual reality and fMRI

项目摘要 形成和提取物体和事件的记忆是人类经验的基础。互补性 内侧颞叶(MTL)内的机制支持对象记忆的形成和检索， 这样，周围皮质支持对单个项目的特征的记忆，而海马体支持 在最初遇到的上下文中回忆对象。尽管大多数关于客体的研究 记忆仅使用视觉刺激进行，一小部分研究表明，视觉对象 与它们特有的声音(例如，狗的叫声和叫声)一起编码的人在以后更容易记住 仅在视觉上编码的对象，或带有无意义或非特征声音的对象。尽管有这些证据和 自然环境中多通道刺激的普遍存在，我们对多感官程度的了解 对记忆及其潜在机制的影响是非常有限的。我最近的行为研究表明 多感官编码的好处是基于对这些对象所处的上下文的改进回忆 都被编码了，这表明多感觉处理独特地参与了绑定这些对象的机制 对周围的信息进行编码。然而，仍然不清楚编码上下文的特征是什么 更好地提取，以及多感觉处理调节了哪些神经机制来实现这一点 利益。拟议的研究通过收集关键证据来解决这一知识差距 确定受多感官影响的物品和周围事件的记忆方面 处理，以及大脑活动模式和涉及到检索这些信息的区域。我们 假设编码时的多感觉处理通过增加可能性来改善物体记忆 对象和上下文将被绑定到情节记忆中以支持稍后对该对象的识别， 而不是通过增强单个对象本身的存储器。此外，我们预测激活的模式 在提取多模式物体的过程中，海马体内也会携带关于环境的信息 这些物品都是在其中编码的。严格测试多感官加工的记忆类型 影响，因此需要评估嵌入在Rich， 自然主义时空语境。为此，我们开发了一个身临其境的、自然的编码任务 在虚拟现实(VR)环境中，其中包含受控但动画的视觉和视听对象。 这一新的方法将使我们能够评估多感觉处理是否影响个体的记忆 对象本身，或者如果这些对象更容易绑定到它们的环境以支持事件的记忆 在上下文中(目标1)。此外，对功能磁共振数据的分析将用于确定MTL的哪些区域和 大脑皮层特别涉及在上下文中看到的多感官物体的记忆(目标2)。最重要的是 这个项目的目标是提高我们对情节记忆是如何在自然主义中形成的知识。 经验和这项工作将有助于我们解释和预测现实世界的行为。

Project Summary Forming and retrieving memories for objects and events is fundamental to human experience. Complementary mechanisms within the medial temporal lobe (MTL) support the formation and retrieval of memories for objects, such that the perirhinal cortex supports memory for features of individual items, and the hippocampus supports recollection of the object within the context it was originally encountered. Although most research on object memory has been conducted using visual stimuli alone, a small body of research has shown that visual objects encoded along with their characteristic sound (e.g., a dog and a bark) are better remembered later on than objects encoded only visually, or with a meaningless or non-characteristic sound. Despite this evidence and the ubiquity of multimodal stimuli within natural environments, our knowledge of the extent of multisensory influences on memory and its underlying mechanisms is very limited. My recent behavioral work showed that the benefits of multisensory encoding are based on improved recollection of the context in which those objects were encoded, suggesting that multisensory processing uniquely engages mechanisms that bind these objects to surrounding information at encoding. However, it remains unclear what features of the encoding context are better retrieved, and what neural mechanisms are modulated by multisensory processing to allow for this benefit. The proposed research addresses this gap in knowledge by collecting the critical evidence to determine the facets of memory for items and surrounding events that are influenced by multisensory processing, and the brain activity patterns and regions that are involved in retrieving such information. We hypothesize that multisensory processing at encoding improves object memory by increasing the likelihood that the object and context will be bound into an episodic memory to support later recognition of that object, rather than by enhancing memory for the individual object itself. Further, we predict that patterns of activation within the hippocampus during retrieval of multimodal objects will also carry information about the environment in which these items were encoded. To rigorously test the type of memory that multisensory processing impacts, there is a need to assess memory for unimodal and multimodal objects that are embedded within rich, naturalistic spatiotemporal context. To this end, we have developed an immersive, naturalistic encoding task within virtual reality (VR) environments, which contain controlled but animated visual and audiovisual objects. This novel approach will allow us to assess whether multisensory processing influences memory for individual objects alone or if these objects are more readily bound to their surroundings to support memory for events within context (Aim 1). Further, analyses of fMRI data will be used to determine which regions of the MTL and cortex are specifically involved in memories of multisensory objects seen in context (Aim 2). The overarching goal of this project is to advance our knowledge of how episodic memories are formed during naturalistic experiences and this work will contribute to our ability to explain and predict real-world behaviors.