Hand Gesture as a Window into the Cognitive and Neural Links between Language and Memory

手势作为了解语言和记忆之间的认知和神经联系的窗口

• 批准号: 9396299
• 负责人: Caitlin Hilverman
• 金额: $ 6.08万
• 依托单位: VANDERBILT UNIVERSITY MEDICAL CENTER
• 依托单位国家: 美国
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2019-08-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/9396299
• 关键词: Address; Affect; Alzheimer' s Disease; Amnesia; Anatomy; Belief; Bilateral; Biological; Biological Neural Networks; Clinical; Cognitive; Communication; Complex; Comprehension; Conscious; Data; Dementia; Development; Disease; Event; Exhibits; Eye; Gestures; Habits; Hand; Health; Height; Hippocampus (Brain); Human; Impairment; Individual; Instruction; Intervention; Knowledge; Language; Language Disorders; Language Tests; Learning; Lesion; Linguistics; Link; Medial; Memory; Memory impairment; Methodology; Methods; Motor; Motor Skills; Movement; Nature; Neuropsychology; Participant; Patients; Population; Process; Production; Psycholinguistics; Rehabilitation therapy; Reporting; Research; Scientist; Semantics; Signal Transduction; Societies; Speech; Structure; System; Techniques; Temporal Lobe; Testing; Time; Training; Update; Water; Work; arm; base; cognitive neuroscience; cognitive skill; experience; flexibility; improved; innovation; language comprehension; language processing; memory process; mental representation; multimodality; neglect; novel; relating to nervous system; skills; theories

Project Summary Memory supports our ability to use language. A number of proposals aim to describe how memory supports our processing, representation, and use of language by linking specific memory systems to different aspects of language (e.g., Caplan & Waters, 1999; Just & Carpenter, 1992; Ullman, 2004). However, these attempts at linking memory and language have largely neglected that language is inherently multimodal; our language is commonly accompanied by meaningful, spontaneous movements of the hands and arms that can communicate information. An explored question is whether and how these hand gestures engage and affect memory processes during language use. Co-speech gestures are tightly linked to spoken language semantically (McNeill, 1992), pragmatically (Kelly et al., 1999), and temporally (Habets et al., 2011). However, gesture does not simply communicate information that is redundant with speech. Rather, gesture, by communicating visually and motorically, can convey information not present in spoken language (McNeill, 1992). This suggests that gesture may reflect content that is qualitatively different from that in speech, potentially communicating aspects or types of memory representations that are different from what is in spoken language. In this proposal, we exploit gesture’s unique communicative nature to examine how it engages and affects specific types of memory during language use. We approach memory from a multiple memory systems framework supported by decades of neuropsychological research, which posits two anatomically and functionally distinct systems: declarative memory – memory for facts and events – and non-declarative memory – memory for skills and habits (e.g., Cohen & Squire, 1980). By receiving training in the lesion method, eye-tracking methodology, and working with clinical populations, I will investigate the hypothesis that hand gesture engages non-declarative learning and memory mechanisms during language use. Under Aim 1, I will test whether gesturing while retelling a narrative affects memory for the content of the narrative, and whether this memory enhancement relies on non-declarative memory mechanisms. Aim 2 focuses on gesture’s integration with spoken language and tests whether this integration can rapidly affect memory representations during language processing. Under Aim 3, I will examine whether gesture production can signal changes in memory states via non-declarative mechanisms, even when this is not evidence in speech. This work will elucidate the memory mechanisms that are engaged in language use and will for the first time bridge theories of memory with theories of multimodal language use, offering a richer and more complete account of the relationship between memory and language. Because concomitant deficits of memory and language are becoming increasingly more common in disorders such as Alzheimer’s Disease and dementia, it is vitally important to have a comprehensive understanding of how they interact. My findings will have implications for instructional and rehabilitative strategies for healthy individuals and individuals with memory impairment to benefit memory functioning and communication.

项目概要 记忆支持我们使用语言的能力。许多提案旨在描述内存如何支持 我们通过将特定的记忆系统与语言的不同方面联系起来来处理、表示和使用语言 语言（例如，Caplan & Waters，1999；Just & Carpenter，1992；Ullman，2004）。然而，这些尝试 将记忆和语言联系起来很大程度上忽略了语言本质上是多模态的；我们的语言是 通常伴随着可以进行交流的有意义的、自发的手和手臂运动 信息。一个探讨的问题是这些手势是否以及如何参与和影响记忆 语言使用过程。共同语音手势在语义上与口语紧密相关（McNeill， 1992）、务实上（Kelly 等人，1999）和时间上（Habets 等人，2011）。然而，手势并不仅仅 传达与言语无关的信息。相反，通过视觉和手势进行交流 通过运动，可以传达口语中不存在的信息（McNeill，1992）。这表明手势 可能反映的内容与言语、潜在的沟通方面或内容有本质上的不同 与口语中不同的记忆表征类型。在这个提案中，我们 利用手势独特的交流性质来检查它如何参与和影响特定类型的记忆 在语言使用过程中。我们从数十年支持的多个内存系统框架中处理内存 神经心理学研究的一部分，它提出了两个解剖学和功能上不同的系统：陈述性 记忆——对事实和事件的记忆——以及非陈述性记忆——对技能和习惯的记忆（例如， 科恩和斯奎尔，1980）。通过接受病变方法、眼球追踪方法和工作方面的培训 对于临床人群，我将研究手势涉及非陈述性学习的假设 以及语言使用过程中的记忆机制。在目标 1 下，我将测试复述时是否可以做手势 叙述会影响对叙述内容的记忆，而这种记忆的增强是否依赖于 非声明性记忆机制。目标 2 侧重于手势与口语和测试的整合 这种整合是否可以在语言处理过程中快速影响记忆表征。在目标 3 下，我 将检查手势的产生是否可以通过非声明性机制发出内存状态变化的信号， 即使这不是言语中的证据。这项工作将阐明参与的记忆机制 语言使用并将首次将记忆理论与多模态语言使用理论联系起来， 对记忆和语言之间的关系提供了更丰富、更完整的描述。因为 伴随的记忆和语言缺陷在以下疾病中变得越来越常见 阿尔茨海默病和痴呆症，全面了解它们是如何发生的至关重要 相互影响。我的发现将对健康的教学和康复策略产生影响 个人和有记忆障碍的个人有利于记忆功能和沟通。