Cross-modal integration of language and face cues in German and French infants

该项目旨在调查婴儿何时以及如何能够以连贯和有意义的表征将音频语音线索和视觉面部线索联系起来。研究这个问题的少数几项研究提供了令人惊讶的声音特征和面部动作之间的联系，但到目前为止还没有找到任何相关的匹配线索。我们的目标是确定这种联系何时出现，它对一种特定语言的具体程度，以及它是否在第一年年底缩小到母语。为了回答这些问题，我们将使用两种语言，法语和德语，它们在语音和韵律方面有很大的差异。相反，德国人和法国人的脸非常相似，这将避免任何“其他种族效应”。在5个实验中，我们将系统地研究听觉和视觉面孔线索之间的联系以及语音和韵律信息的作用。使用与德语或法语属于相同或不同节奏类别的语言，我们还打算调查视觉-听觉联系的概括性。我们将研究是否可以将这种联系推广到属于同一节奏类的语言，如英语(如德语)和西班牙语(如法语)。该项目的跨语言方面需要法德合作。这使得从单一语言家庭招募和学习学习法语和德语的婴儿成为可能。以前的大多数跨语言研究关注的是非常不同的主题，只对来自一种语言的参与者进行了不同语言刺激的测试。然而，计划中的项目将利用交叉设计，并利用法德合作的独特可能性，通过在联合项目中使用相同的语言刺激来研究具有不同母语的参与者。

The project aims to investigate when and how infants are able to link audio speech cues and visual face cues in a coherent and meaningful representation. The few studies investigating this question provided surprising links between voice characteristics and facial movements but so far failed to identify any relevant matching cues. We aim to determine when this link emerges, how specific it is to a particular language and whether it narrows to the native language toward the end of the first year. To address those questions we will use two languages, French and German, that greatly differ in terms of phonetics and prosody. On the contrary, German and French faces are rather similar which will avoid any "other-race effect". In 5 experiments, we will systematically investigate the proposed link between auditory and visual face cues and the role of phonetic and prosodic information. Using languages which belong to the same or different rhythmic classes as German or French, we also intend to investigate the generalization of the visual-auditory association. We will study whether the link can be generalized to languages belonging to the same rhythmic class, such as English (like German) and Spanish (like French). The cross-linguistic aspect of the project requires a French-German collaboration. This makes it possible to recruit and study French and German learning infants from monolingual families. Most previous cross linguistic studies focused on very different topics and only tested participants from one language with different language Stimuli. The intended project, however, will take advantage of a cross-over design and use the unique possibility of a French-German collaboration to study participants with different native languages by using the same language Stimuli in a joint project.