Empirical and computational study of iconic gesture-speech integration and its development in preschool children (EcoGest)

学龄前儿童标志性手势-语音整合及其发展的实证和计算研究（EcoGest）

• 批准号: 322326507
• 负责人: Professorin Dr. Friederike Kern
• 金额: --
• 依托单位: Fachbereich Ingenieurwissenschaften und Mathematik
• 依托单位国家: 德国
• 项目类别: Research Grants
• 财政年份: 2017
• 资助国家: 德国
• 起止时间: 2016-12-31 至 2021-12-31
• 项目状态: 已结题
• 来源: https://gepris.dfg.de/gepris/projekt/322326507?language=en
• 关键词: Empirical; computational; study; iconic; gesture

Gesture and speech form an integrated communicative system. This integration has been investigated for the purpose of communication as well as learning. Recent research attempting to explain and model this integration focus on how gesture influences speech, and how gesture -- as a special form of action -- forces the speaker to incorporate sensorimotor information into the representation. In these cognitive approaches, the fact that gesture is employed in various communicative genres and thus interacts with speech for different discursive purposes is disregarded. Yet types of communicative genres have been shown to bear different cognitive and communicative requirements on the speaker and hearer: For example, a narration requires different structural knowledge and verbal skills from a speaker than an explanation. While for competent speakers, the appearance of gestures in different communicative genres might occur fluent, for children learning to cope with the variety of discursive contexts, the role of gesture for thinking and communicating might be better observable: One possibility is that for particular genres, gestures are required resources for depicting strategies facilitating mutual understanding, and children discover them with increasing experience in interaction and with growing cognitive and linguistic competencies. Another -- not exclusive -- possibility is that gesture plays an important role on the way to master increasingly changing communicative requirements, which becomes visible when children learn to produce different communicative genres. Thus, in addition to sensorimotor information that is recruited for the act of speaking, gesture might be an integral resource of speaking in relation to different communicative purposes. This possibility raises new questions for models of gesture-speech integration: Is gesture used in dependence of the experience in communicative genre and if so how is the pragmatic aspect incorporated into the production of speech? In our proposed project, we apply qualitative and quantitative analyses as well as computational evaluation methods in investigating longitudinally how the integration of gesture and speech develops in 4 and 5 years old preschool children and how context (meaning the communicative genre and physical surroundings) relates to the different gestural techniques that children use in interaction. The results of our project will provide a crucial contribution towards a comprehensive developmental theory of gesture and speech integration involving cognitive and pragmatic aspects of speaking.

手势和言语构成了一个完整的交际系统。这种整合已被调查的沟通以及学习的目的。最近的研究试图解释和建模这种整合集中在手势如何影响语音，以及手势-作为一种特殊的动作形式-如何迫使扬声器将感觉运动信息纳入表示。在这些认知方法中，手势被用于各种交际体裁，从而与言语相互作用以实现不同的话语目的这一事实被忽视。然而，不同类型的交际体裁对说话者和听话者的认知和交际要求不同：例如，叙述需要说话者提供不同的结构知识和语言技能。虽然对于有能力的演讲者来说，不同交际体裁中的手势可能会出现流畅的外观，但对于学习科普各种话语背景的儿童来说，手势在思维和交流中的作用可能更容易观察到：一种可能性是，对于特定的体裁，手势是描述促进相互理解的策略所必需的资源，孩子们在互动中积累了越来越多的经验，认知和语言能力也在不断提高。另一种--但不是唯一的--可能性是，手势在掌握日益变化的交际要求的过程中发挥着重要作用，当孩子们学习产生不同的交际类型时，这一点就变得显而易见。因此，除了用于说话行为的感觉运动信息之外，手势可能是与不同交际目的相关的说话的不可或缺的资源。这种可能性提出了新的问题的模型的手势语音整合：手势的使用依赖的经验，在交际体裁，如果是这样的话，是如何语用方面纳入生产的讲话？在我们提出的项目中，我们采用定性和定量分析以及计算评估方法，纵向调查如何整合的手势和言语的发展，在4和5岁的学龄前儿童和如何上下文（即交际类型和物理环境）涉及到不同的手势技术，儿童在互动中使用。我们的项目的结果将提供一个全面的发展理论的手势和语音整合，涉及认知和语用方面的发言的重要贡献。