Hand Gesture as a Window into the Cognitive and Neural Links between Language and Memory

项目摘要 记忆支持我们使用语言的能力。许多建议旨在描述内存如何支持 通过将特定的记忆系统与语言的不同方面联系起来， 语言（例如，Caplan &沃茨，1999年; Just & Carpenter，1992年; Ullman，2004年）。然而，这些尝试 将记忆和语言联系起来，在很大程度上忽略了语言本质上是多模态的;我们的语言是 通常伴随着有意义的、自发的手和手臂的运动， 信息.一个探索的问题是这些手势是否以及如何参与和影响记忆 在语言使用过程中。共语手势在语义上与口语紧密相连（麦克尼尔， 1992年），务实（凯利等人，1999）和时间上（Habets等人，2011年）。然而，手势并不简单 用语言传达多余的信息。相反，手势，通过视觉交流， 在运动上，可以传达口语中不存在的信息（麦克尼尔，1992）。这表明， 可以反映与语音中的内容在质量上不同的内容，潜在地传达方面或 与口头语言不同的记忆表征类型。在本提案中，我们 利用手势独特的交流性质来研究它如何参与和影响特定类型的记忆 在语言使用中。我们从几十年来支持的多记忆系统框架来处理记忆 神经心理学研究，其中假定两个解剖学和功能不同的系统： 记忆-事实和事件的记忆-和非陈述性记忆-技能和习惯的记忆（例如， Cohen & Squire，1980）。通过接受损伤方法、眼动追踪方法的培训， 在临床人群中，我将研究手势参与非陈述性学习的假设 和记忆机制。在目标1下，我将测试在复述一个 叙事会影响对叙事内容的记忆，这种记忆增强是否依赖于 非陈述性记忆机制目标2侧重于手势与口语和测试的整合 这种整合是否能在语言处理过程中迅速影响记忆表征。在目标3下， 将检查手势产生是否可以通过非声明机制来表示记忆状态的变化， 即使这在言语中不是证据。这项工作将阐明记忆机制，参与 语言使用和意志第一次将记忆理论与多模态语言使用理论联系起来， 对记忆和语言之间的关系提供了更丰富、更完整的解释。因为 伴随的记忆和语言缺陷在以下疾病中变得越来越常见， 阿尔茨海默病和痴呆症，这是至关重要的是要有一个全面的了解，他们如何 互动.我的研究结果将对健康儿童的教学和康复策略产生影响。 个人和个人的记忆障碍，以有利于记忆功能和沟通。

Project Summary Memory supports our ability to use language. A number of proposals aim to describe how memory supports our processing, representation, and use of language by linking specific memory systems to different aspects of language (e.g., Caplan & Waters, 1999; Just & Carpenter, 1992; Ullman, 2004). However, these attempts at linking memory and language have largely neglected that language is inherently multimodal; our language is commonly accompanied by meaningful, spontaneous movements of the hands and arms that can communicate information. An explored question is whether and how these hand gestures engage and affect memory processes during language use. Co-speech gestures are tightly linked to spoken language semantically (McNeill, 1992), pragmatically (Kelly et al., 1999), and temporally (Habets et al., 2011). However, gesture does not simply communicate information that is redundant with speech. Rather, gesture, by communicating visually and motorically, can convey information not present in spoken language (McNeill, 1992). This suggests that gesture may reflect content that is qualitatively different from that in speech, potentially communicating aspects or types of memory representations that are different from what is in spoken language. In this proposal, we exploit gesture’s unique communicative nature to examine how it engages and affects specific types of memory during language use. We approach memory from a multiple memory systems framework supported by decades of neuropsychological research, which posits two anatomically and functionally distinct systems: declarative memory – memory for facts and events – and non-declarative memory – memory for skills and habits (e.g., Cohen & Squire, 1980). By receiving training in the lesion method, eye-tracking methodology, and working with clinical populations, I will investigate the hypothesis that hand gesture engages non-declarative learning and memory mechanisms during language use. Under Aim 1, I will test whether gesturing while retelling a narrative affects memory for the content of the narrative, and whether this memory enhancement relies on non-declarative memory mechanisms. Aim 2 focuses on gesture’s integration with spoken language and tests whether this integration can rapidly affect memory representations during language processing. Under Aim 3, I will examine whether gesture production can signal changes in memory states via non-declarative mechanisms, even when this is not evidence in speech. This work will elucidate the memory mechanisms that are engaged in language use and will for the first time bridge theories of memory with theories of multimodal language use, offering a richer and more complete account of the relationship between memory and language. Because concomitant deficits of memory and language are becoming increasingly more common in disorders such as Alzheimer’s Disease and dementia, it is vitally important to have a comprehensive understanding of how they interact. My findings will have implications for instructional and rehabilitative strategies for healthy individuals and individuals with memory impairment to benefit memory functioning and communication.