Hand Gesture as a Window into the Cognitive and Neural Links between Language and Memory

项目摘要 记忆支持我们使用语言的能力。许多提案旨在描述内存如何支持 通过将特定的记忆系统链接到语言的不同方面，我们对语言的处理、表示和使用 语言(例如，Caplan&Waters，1999；Just&Carpenter，1992；Ullman，2004)。然而，这些尝试 将记忆和语言联系在一起在很大程度上忽略了语言天生的多模式；我们的语言是 通常伴随着有意义的、自发的可以交流的手和手臂的运动 信息。一个被探究的问题是这些手势是否以及如何参与并影响记忆 语言使用过程中的过程。协同语音手势在语义上与口语紧密相连(McNeill， 1992年)、语用(Kelly等人，1999年)和时间(Habets等人，2011年)。然而，手势并不是简单的 用语言交流多余的信息。相反，手势，通过视觉交流和 在汽车上，可以传达口语中不存在的信息(McNeill，1992)。这表明这一手势 可能反映与语音、潜在的通信方面或 不同于口语的记忆表征类型。在这项提案中，我们 利用手势的独特交流特性来研究它是如何参与和影响特定类型的记忆的 在使用语言的过程中。我们从一个由数十年支持的多内存系统框架来处理内存 神经心理学研究，它假设了两个在解剖学和功能上截然不同的系统：陈述性 记忆--事实和事件的记忆--和非陈述性记忆--技能和习惯的记忆(例如， Cohen&Squire，1980)。通过接受损伤方法、眼球跟踪方法和工作方面的培训 在临床人群中，我将调查手势参与非陈述性学习的假设 以及语言使用过程中的记忆机制。在目标1下，我将测试在复述的同时打手势是否 叙事影响对叙事内容的记忆，以及这种记忆增强是否依赖于 非声明性内存机制。目标2侧重于手势与口语和测试的整合 这种整合是否会在语言处理过程中迅速影响内存表示。在目标3下，我 将检查手势产生是否可以通过非声明性机制发出存储器状态变化的信号， 即使这不是口头上的证据。这项工作将阐明参与的记忆机制 语言使用和Will首次将记忆理论与多模式语言使用理论联系起来， 对记忆和语言之间的关系提供了更丰富、更完整的解释。因为 伴随而来的记忆和语言缺陷在以下疾病中越来越常见 对于阿尔茨海默病和痴呆症，全面了解它们是如何发生的至关重要 互动。我的发现将对健康的教学和康复策略产生影响 有利于记忆功能和沟通的个人和有记忆障碍的个人。

Project Summary Memory supports our ability to use language. A number of proposals aim to describe how memory supports our processing, representation, and use of language by linking specific memory systems to different aspects of language (e.g., Caplan & Waters, 1999; Just & Carpenter, 1992; Ullman, 2004). However, these attempts at linking memory and language have largely neglected that language is inherently multimodal; our language is commonly accompanied by meaningful, spontaneous movements of the hands and arms that can communicate information. An explored question is whether and how these hand gestures engage and affect memory processes during language use. Co-speech gestures are tightly linked to spoken language semantically (McNeill, 1992), pragmatically (Kelly et al., 1999), and temporally (Habets et al., 2011). However, gesture does not simply communicate information that is redundant with speech. Rather, gesture, by communicating visually and motorically, can convey information not present in spoken language (McNeill, 1992). This suggests that gesture may reflect content that is qualitatively different from that in speech, potentially communicating aspects or types of memory representations that are different from what is in spoken language. In this proposal, we exploit gesture’s unique communicative nature to examine how it engages and affects specific types of memory during language use. We approach memory from a multiple memory systems framework supported by decades of neuropsychological research, which posits two anatomically and functionally distinct systems: declarative memory – memory for facts and events – and non-declarative memory – memory for skills and habits (e.g., Cohen & Squire, 1980). By receiving training in the lesion method, eye-tracking methodology, and working with clinical populations, I will investigate the hypothesis that hand gesture engages non-declarative learning and memory mechanisms during language use. Under Aim 1, I will test whether gesturing while retelling a narrative affects memory for the content of the narrative, and whether this memory enhancement relies on non-declarative memory mechanisms. Aim 2 focuses on gesture’s integration with spoken language and tests whether this integration can rapidly affect memory representations during language processing. Under Aim 3, I will examine whether gesture production can signal changes in memory states via non-declarative mechanisms, even when this is not evidence in speech. This work will elucidate the memory mechanisms that are engaged in language use and will for the first time bridge theories of memory with theories of multimodal language use, offering a richer and more complete account of the relationship between memory and language. Because concomitant deficits of memory and language are becoming increasingly more common in disorders such as Alzheimer’s Disease and dementia, it is vitally important to have a comprehensive understanding of how they interact. My findings will have implications for instructional and rehabilitative strategies for healthy individuals and individuals with memory impairment to benefit memory functioning and communication.