Influence of spoken text on cognitive processing of complex pictorial presentations

基于多媒体学习模型（Glaser & Schwan，2015; Mayer，2014）和对图片和场景感知的心理学研究结果（Huettig & Altmann，2005），计划中的项目研究了伴随的音频文本对复杂图片内容的感知和处理以及相关知识获取过程的影响。在四个实验室实验中，将分析音频文本在多大程度上（1）系统地减少图片的解释开放性，（2）促进被忽视的图片元素的感知和处理，（3）感知和认知相关的图片元素，（4）有助于图片内容的双重编码。研究对象是关于历史事件的复杂而详细的历史绘画，通常用于传达历史知识（Popp，2013）。在这四项研究中，听觉介绍文本的影响，命名的显着的图片元素，命名的图片元素之间的关系，以及文本结构的影响，注意分配（凝视连贯性，固定时间，过渡量，通过眼动跟踪测量）和知识获取（自由视觉回忆，文本-图片-关系，迁移，记忆相关的个体间一致性）将被检查。

Based on models of multimedia learning (Glaser & Schwan, 2015; Mayer, 2014) and psychological findings on picture and scene perception (Huettig & Altmann, 2005), the planned project examines the influence of accompanying audio texts on the perception and processing of complex pictorial contents and related knowledge acquisition processes. In four laboratory experiments it will be analyzed to which extent audio texts (1) systematically reduce the interpretational openness of pictures, (2) foster the perception and processing of neglected pictorial elements, (3) perceptually and cognitively relate pictorial elements, and (4) contribute to the dual coding of the pictorial contents. Research objects are complex and detailed history paintings about historical events, which are typically used to communicate historical knowledge (Popp, 2013). In the four studies, the influence of auditive intro texts, named salient pictorial elements, named relations between pictorial elements as well as the influence of text structure on the allocation of attention (gaze coherence, fixation times, amount of transitions, measured by eye tracking) and knowledge acquisition (free visual recall, text-picture-relation, transfer, memory related interindividual consistency) will be examined.