CAREER: Mechanisms of the Impact of Gesture Observation on Learning

在听演讲的同时观察相关的手势，结合多种感官来增强学习。然而，在听演讲时观察到的手势在促进学习的程度上有所不同，目前人们对为什么会出现这种情况知之甚少。这个职业奖研究了通过手势传达的信息及其与言语的关系如何影响从发音和意义不同的新第二语言中学习单词。通过研究大脑如何处理以这些方式不同的手势学习的这些单词，它将深入了解为什么不同类型的手势对学习的影响不同。本CAREER奖的研究目标是：(1)确定通过手势传达的信息及其在学习中与言语的一致性如何影响通过言语传达的学习内容的后续加工；(2)表征手势在学习前对言语传达信息的敏感性及其与言语的一致性如何影响手势对言语传达学习内容的学习和后续加工的影响；(3)确定多模态和语言处理的脑区如何影响手势-语音整合以及通过语音传递的学习内容的后续处理。该研究方法通过观察在学习过程中观察与非母语语音和单词指涉物一致或不一致的手势如何通过准确性和延迟、事件相关电位（ERPs）和经颅直接刺激（tDCS）对习得的第二语言（L2）单词的后续加工产生影响，从而达到这些目标。在此过程中，它将有助于发展手势对学习的影响理论，该理论适用于不同的内容领域，具体说明手势对学习影响的认知机制的神经基础。该职业奖的教育目标将把基于探究的学习利用ERP研究方法整合到教育神经科学课程中；吸引来自不同背景的学生参与拟议的研究和教育活动；并促进研究结果在第二语言教学环境中的复制和翻译。总之，这些活动将有助于弥合教育神经科学研究与多种教育背景下的教学实践之间的差距。该项目由学习科学和增强智能计划以及刺激竞争研究的既定计划（EPSCoR）共同资助。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Observing relevant hand gestures while listening to speech incorporates multiple senses that enhance learning. However, gestures observed when listening to speech vary in the extent to which they enhance learning, and it is currently poorly understood why this is the case. This CAREER award examines how the information conveyed via gesture and its relationship to speech affects the learning of words from a novel second language differing in pronunciation and meaning. By examining how the brain processes such words learned with gestures differing in these ways, it will provide insight into why various types of gestures affect learning differently.The research objectives of this CAREER award are to (1) determine how information conveyed via gesture and its congruence with speech at learning affects subsequent processing of learned content conveyed via speech; (2) characterize how sensitivity to information conveyed via gesture and its congruence with speech prior to learning affects the impact of gesture on learning and subsequent processing of learned content conveyed via speech; and (3) identify how brain regions subserving multimodal and language processing affect gesture-speech integration and subsequent processing of learned content conveyed via speech. The research approach achieves these objectives by examining how observing gestures congruent and incongruent with non-native speech sounds and word referents at learning affects subsequent processing of learned second language (L2) words via accuracy and latency, event-related potentials (ERPs), and transcranial direct stimulation (tDCS), which causally affects brain activity. In doing so, it will contribute to the development of a theory of gesture’s impact on learning applicable across diverse content areas specifying the neural substrates of the cognitive mechanisms of gesture’s impact on learning. The educational objectives of this CAREER award will integrate scaffolded inquiry-based learning utilizing ERP research methods into the educational neuroscience curriculum; engage students from diverse backgrounds students in the proposed research and educational activities; and advance replication and translation of findings in L2 pedagogical contexts. Together, these activities will help bridge the gap between educational neuroscience research and pedagogical practice in multiple educational contexts. This project is jointly funded by the Science of Learning and Augmented Intelligence Program and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.