Coreference Processing in Gesture and Speech

描述（由申请人提供）：语言理解和生产的一个必要操作是共指，这包括在记忆中跟踪概念，更新它们，并在不同的共指形式之间形成正确的联系，通常是长距离的。语篇理解中的共指过程已经得到了很好的研究，而话语和韵律线索对言语和手势模式的相关贡献却很少得到关注。本提案寻求资金来研究语音手势理解和产生中的共指过程，以便更完整地模拟成人最终状态语言系统的组织和机制。将测试四个假设。(1)韵律重音获取并利用了与文本理解相同的潜在记忆机制和组织原则。(2)手势可以用来标记先行词，这种共指手势与言语是共表达的。(3)指涉手势与韵律重音和语篇理解具有相同的记忆机制和组织原则。(4)手势以运动-动力学格式编码先行表征。

DESCRIPTION (provided by applicant): A necessary operation for language comprehension and production is coreference, which involves keeping track of concepts in memory, updating them, and forming correct links in the mind between different coreferring forms, often over long distances. The process of coreference in text comprehension has been well studied, while the contribution of discourse and prosodic cues relevant for the speech and gesture modalities has received considerably less attention. This proposal seeks funding to investigate the process of coreference in speech-gesture comprehension and production in order to model the organization and mechanisms of the adult, end-state, language system more completely. Four hypotheses will be tested. (1) Prosodic stress accesses and makes use of the same underlying memory mechanism(s) and organizing principles as found for text comprehension. (2) Gesture can be used to mark antecedents and this coreferent gesture is co-expressive with speech. (3) Coreferring gestures also make use of the same underlying memory mechanism(s) and organizing principles as prosodic stress and text comprehension. (4) Gesture codes antecedent representations in a motor-kinetic format.