Influence of spoken text on cognitive processing of complex pictorial presentations

点击翻译按钮获取中文摘要

Based on models of multimedia learning (Glaser & Schwan, 2015; Mayer, 2014) and psychological findings on picture and scene perception (Huettig & Altmann, 2005), the planned project examines the influence of accompanying audio texts on the perception and processing of complex pictorial contents and related knowledge acquisition processes. In four laboratory experiments it will be analyzed to which extent audio texts (1) systematically reduce the interpretational openness of pictures, (2) foster the perception and processing of neglected pictorial elements, (3) perceptually and cognitively relate pictorial elements, and (4) contribute to the dual coding of the pictorial contents. Research objects are complex and detailed history paintings about historical events, which are typically used to communicate historical knowledge (Popp, 2013). In the four studies, the influence of auditive intro texts, named salient pictorial elements, named relations between pictorial elements as well as the influence of text structure on the allocation of attention (gaze coherence, fixation times, amount of transitions, measured by eye tracking) and knowledge acquisition (free visual recall, text-picture-relation, transfer, memory related interindividual consistency) will be examined.