Hand Gesture as a Window into the Cognitive and Neural Links between Language and Memory

点击翻译按钮获取中文摘要

Project Summary Memory supports our ability to use language. A number of proposals aim to describe how memory supports our processing, representation, and use of language by linking specific memory systems to different aspects of language (e.g., Caplan & Waters, 1999; Just & Carpenter, 1992; Ullman, 2004). However, these attempts at linking memory and language have largely neglected that language is inherently multimodal; our language is commonly accompanied by meaningful, spontaneous movements of the hands and arms that can communicate information. An explored question is whether and how these hand gestures engage and affect memory processes during language use. Co-speech gestures are tightly linked to spoken language semantically (McNeill, 1992), pragmatically (Kelly et al., 1999), and temporally (Habets et al., 2011). However, gesture does not simply communicate information that is redundant with speech. Rather, gesture, by communicating visually and motorically, can convey information not present in spoken language (McNeill, 1992). This suggests that gesture may reflect content that is qualitatively different from that in speech, potentially communicating aspects or types of memory representations that are different from what is in spoken language. In this proposal, we exploit gesture’s unique communicative nature to examine how it engages and affects specific types of memory during language use. We approach memory from a multiple memory systems framework supported by decades of neuropsychological research, which posits two anatomically and functionally distinct systems: declarative memory – memory for facts and events – and non-declarative memory – memory for skills and habits (e.g., Cohen & Squire, 1980). By receiving training in the lesion method, eye-tracking methodology, and working with clinical populations, I will investigate the hypothesis that hand gesture engages non-declarative learning and memory mechanisms during language use. Under Aim 1, I will test whether gesturing while retelling a narrative affects memory for the content of the narrative, and whether this memory enhancement relies on non-declarative memory mechanisms. Aim 2 focuses on gesture’s integration with spoken language and tests whether this integration can rapidly affect memory representations during language processing. Under Aim 3, I will examine whether gesture production can signal changes in memory states via non-declarative mechanisms, even when this is not evidence in speech. This work will elucidate the memory mechanisms that are engaged in language use and will for the first time bridge theories of memory with theories of multimodal language use, offering a richer and more complete account of the relationship between memory and language. Because concomitant deficits of memory and language are becoming increasingly more common in disorders such as Alzheimer’s Disease and dementia, it is vitally important to have a comprehensive understanding of how they interact. My findings will have implications for instructional and rehabilitative strategies for healthy individuals and individuals with memory impairment to benefit memory functioning and communication.