Touch to learn: How sensory cues impact word segmentation and learning

点击翻译按钮获取中文摘要

Project summary Infants experience language in a rich multimodal world in which sounds, sights, smells, tastes, and touches occur together. How do these modalities, and their presence in the input, contribute to fundamental aspects of lexical acquisition? We explore how infants exploit multi- compared with uni-modal input as they segment wordforms from continuous speech and map meanings to wordforms. We hypothesize that the ability to make use of multimodal information including touch (a fundamental social signal) in the input facilitates infants' language learning by contributing information relevant to the alignment of word boundaries and the association of word meaning to wordforms. In Aim 1 (Experiments 1a-c) we determine whether sensory input that incorporates more modalities aligned with speech input, results in improved segmentation of the speech stream and improved attention to word edges for infant learners. In Aim 2 (Experiments 2a and 2b) we determine whether sensory input that incorporates more modalities associated with objects results in improved word learning for language learners. Our approach builds on our preliminary work showing that (a) caregiver touch+speech alignment and co- occurrence is common in interactions with infants and (b) infants use multisensory information in early language learning tasks. The design exploits behavioral and physiological measures to examine whether the alignment (Aim 1) and/or association (Aim 2) of multimodal touch, speech, and visual cues influence language learning in typically developing children. The proposed research is significant for its clinical and theoretical implications. First, it is clinically significant since exploring the role of multimodal cues on language learning may impact clinical approaches since current language therapies or interventions concerning the input tend to emphasize auditory alone or auditory+visual domains, but not tactile ones. Thus, discoveries concerning the child's use of multimodal information which includes touch in language learning could lead to new avenues for early intervention for infants and toddlers showing delayed language acquisition or who are at-risk of language disorders. Second, this work is theoretically significant since it explores how multimodal information may help the learner at different levels of representation (wordform representations, semantic representations) in ways that could constrain the language learning problem. The findings from this research will challenge our views concerning what kinds of informational streams infants attend to when learning language, and thus influence intervention and parenting practices.