CAREER: Mechanisms of the Impact of Gesture Observation on Learning

在听演讲时观察相关的手势结合了多种感官，从而增强了学习。然而，在听演讲时观察到的手势在促进学习的程度上有所不同，目前人们对为什么会出现这种情况知之甚少。这一职业奖考察了通过手势传达的信息及其与言语的关系如何影响从发音和意义不同的新的第二语言中学习单词。本职业奖的研究目标是：(1)确定通过手势传递的信息及其与言语的一致性如何影响随后通过言语传递的信息；(2)表征对通过手势传递的信息的敏感性及其在学习前与言语的一致性如何影响手势对学习的影响以及随后通过言语传递的学习内容的处理；以及(3)确定参与多模式和语言处理的大脑区域如何影响手势-言语整合以及后续通过言语传递的学习内容的处理。该研究方法通过考察在学习过程中观察手势与非母语语音和词指代的一致和不一致如何通过准确性和潜伏期、事件相关电位(ERPs)和经颅直接刺激(TDC)影响学习的第二语言(L2)单词的后续加工，从而影响大脑活动，从而实现这些目标。通过这样做，它将有助于发展一种适用于不同内容领域的手势对学习的影响理论，该理论指明了手势对学习影响的认知机制的神经基础。这一职业奖项的教育目标将把利用企业资源规划研究方法的脚手架探究式学习纳入教育神经科学课程；让来自不同背景的学生参与拟议的研究和教育活动；并在第二语言教学背景下促进研究成果的复制和翻译。总之，这些活动将有助于在多种教育背景下弥合教育神经科学研究和教学实践之间的差距。该项目由学习科学和增强智能计划和既定的激励竞争研究计划(EPSCoR)共同资助。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Observing relevant hand gestures while listening to speech incorporates multiple senses that enhance learning. However, gestures observed when listening to speech vary in the extent to which they enhance learning, and it is currently poorly understood why this is the case. This CAREER award examines how the information conveyed via gesture and its relationship to speech affects the learning of words from a novel second language differing in pronunciation and meaning. By examining how the brain processes such words learned with gestures differing in these ways, it will provide insight into why various types of gestures affect learning differently.The research objectives of this CAREER award are to (1) determine how information conveyed via gesture and its congruence with speech at learning affects subsequent processing of learned content conveyed via speech; (2) characterize how sensitivity to information conveyed via gesture and its congruence with speech prior to learning affects the impact of gesture on learning and subsequent processing of learned content conveyed via speech; and (3) identify how brain regions subserving multimodal and language processing affect gesture-speech integration and subsequent processing of learned content conveyed via speech. The research approach achieves these objectives by examining how observing gestures congruent and incongruent with non-native speech sounds and word referents at learning affects subsequent processing of learned second language (L2) words via accuracy and latency, event-related potentials (ERPs), and transcranial direct stimulation (tDCS), which causally affects brain activity. In doing so, it will contribute to the development of a theory of gesture’s impact on learning applicable across diverse content areas specifying the neural substrates of the cognitive mechanisms of gesture’s impact on learning. The educational objectives of this CAREER award will integrate scaffolded inquiry-based learning utilizing ERP research methods into the educational neuroscience curriculum; engage students from diverse backgrounds students in the proposed research and educational activities; and advance replication and translation of findings in L2 pedagogical contexts. Together, these activities will help bridge the gap between educational neuroscience research and pedagogical practice in multiple educational contexts. This project is jointly funded by the Science of Learning and Augmented Intelligence Program and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.