Cross-modal processing and lexical development

在许多情况下，人们必须同时处理所呈现的听觉和视觉

In many situations people have to process simultaneously presented auditory and visual information, and this ability is especially important for lexical development where auditorily presented words often co-occur with visually presented objects and scenes. We argue that processing of cross-modal input depends on two critical parameters: (a) the speed of orienting to a modality relative to the competing modality and (b) the dwell time of attention. We also argue that both parameters undergo developmental change, which results in faster and more efficient processing of cross-modal input. Based on these ideas, we propose a project that (a) estimates the critical parameters of auditory-visual processing (Study 1) and (b) uses these parameters to understand the ability to extract the structure from visual input, such as category learning or learning of correlations (Study 2). The major innovation of the proposed research is the theoretical proposal linking some important aspects of cognitive development to more basic mechanism(s) underlying cross-modal processing. The proposed project has two specific aims. Specific aim 1 is to determine how auditory input affects attention allocated to corresponding visual input and to develop a modality-independent measure capable of examining auditory, visual and cross-modal processing. Specific aim 2 is to directly test the ability of attentional mechanisms underlying crossmodal processing to account for effects of auditory input on the ability to extract statistical structure from visual input. The experiments will be guided by our hypothesis that the attentional mechanism underlying cross-modal processing can account for performance on higher-order cognitive tasks, which require processing of visual structure such as word learning, category learning and learning of multiple contingencies. Each study will include multiple experiments with 8-, 12- and 16-monthold infants and will include a variety of methodologies. The proposed research is important because it will generate new knowledge affecting our understanding of the development of attention and its role in cross-modal processing, thus revealing some basic mechanisms of early lexical and cognitive development. This knowledge may also have broader impact on the clinical community by elucidating factors underlying Specific Language Impairments (SLI).