Cross-modal integration of language and face cues in German and French infants

该项目旨在调查婴儿何时以及如何能够将语音线索和视觉面部线索以连贯和有意义的方式联系起来。调查这个问题的少数研究提供了声音特征和面部运动之间令人惊讶的联系，但到目前为止还没有发现任何相关的匹配线索。我们的目标是确定这种联系何时出现，它对一种特定语言有多具体，以及在第一年末是否缩小到母语。为了回答这些问题，我们将使用法语和德语这两种语言，这两种语言在语音和韵律方面都有很大的不同。相反，德国人和法国人的脸非常相似，这就避免了任何“其他种族效应”。在5个实验中，我们将系统地研究听觉和视觉面部线索之间的联系以及语音和韵律信息的作用。使用与德语或法语属于相同或不同节奏类的语言，我们也打算调查视觉听觉联想的泛化。我们将研究这种联系是否可以推广到属于同一节奏类的语言，如英语（如德语）和西班牙语（如法语）。该项目的跨语言方面需要法德合作。这使得招募和研究来自单语家庭的法语和德语学习婴儿成为可能。以前的跨语言研究大多集中在非常不同的主题上，并且只对来自一种语言的参与者进行不同语言刺激的测试。然而，这个计划中的项目将利用交叉设计的优势，利用法德合作的独特可能性，通过在一个联合项目中使用相同的语言刺激来研究母语不同的参与者。

The project aims to investigate when and how infants are able to link audio speech cues and visual face cues in a coherent and meaningful representation. The few studies investigating this question provided surprising links between voice characteristics and facial movements but so far failed to identify any relevant matching cues. We aim to determine when this link emerges, how specific it is to a particular language and whether it narrows to the native language toward the end of the first year. To address those questions we will use two languages, French and German, that greatly differ in terms of phonetics and prosody. On the contrary, German and French faces are rather similar which will avoid any "other-race effect". In 5 experiments, we will systematically investigate the proposed link between auditory and visual face cues and the role of phonetic and prosodic information. Using languages which belong to the same or different rhythmic classes as German or French, we also intend to investigate the generalization of the visual-auditory association. We will study whether the link can be generalized to languages belonging to the same rhythmic class, such as English (like German) and Spanish (like French). The cross-linguistic aspect of the project requires a French-German collaboration. This makes it possible to recruit and study French and German learning infants from monolingual families. Most previous cross linguistic studies focused on very different topics and only tested participants from one language with different language Stimuli. The intended project, however, will take advantage of a cross-over design and use the unique possibility of a French-German collaboration to study participants with different native languages by using the same language Stimuli in a joint project.