权益分类	功能权益	普通用户	{{item.name}}会员
{{category.name}}	{{benefitItem.name}}

Words and pictures: understanding how people gather information conveyed jointly through text and image in comics

文字和图片：了解人们如何收集通过漫画中的文字和图像共同传达的信息

基本信息

批准号：
ES/M007081/1
负责人：
Benjamin Tatler
金额：
$ 36.51万
依托单位：
University of Aberdeen
依托单位国家：
英国
项目类别：
Research Grant
财政年份：
2015
资助国家：
英国
起止时间：
2015 至无数据
项目状态：
已结题

来源：
https://gtr.ukri.org/projects?ref=ES%2FM007081%2F1
关键词：
Words pictures understanding how people

项目摘要

Word and image are used together to convey a wide range of information that we need to complete our behavioural goals, including information vital to our safety (safety signage), our understanding and interaction with the world (instruction manuals, maps, directional signage), the choices we make (advertising) and entertainment (cartoons, comics). Not only are words and images combined in these situations, but it is often only through understanding the combination of word and image - the information that they jointly convey - that we understand what is being communicated to us. Despite the pervasive nature of word-image combinations in the world around us and our reliance on understanding the information that they convey, little is understood about how we view and understand this class of stimulus. Most of our knowledge is limited to how we view and understand words and passages of text or images alone. From these previous approaches, detailed understanding has been developed, and in many ways the manner in which we inspect images and words is very different: with different factors influencing what we look at, and perhaps even different underlying processing styles and cognitive processes when we are reading compared to when we are viewing images. These very different accounts of viewing and processing words and images raise questions about how we gather and understand stimuli that contain both word and image. The proposed work aims to provide key insights into this currently under-developed field of research. Three key questions must be addressed to better understand how we inspect and understand stimuli that combine word and image: (1) What strategies do we use to gather information from word-image combinations and do these change depending on the relationship between the words and image? That is, do we vary how we view these stimuli when the relative amount of information conveyed by the words compared to the image changes? (2) How do we understand and respond to space in word-image combinations? Space between words conveys very different information than space between objects in an image. (3) How much information do we process before looking directly at a word-image combination? We process some aspects of words and images in peripheral vision, but the level to which we do so is controversial. Comics offer an ideal medium for addressing these three questions because they use word-image combinations to convey a story to the reader, over several panels and pages, which must be understood together for the reader to understand the conveyed information. Critically, it is the combination of word and image in comics that conveys the story to the reader, with neither word nor image alone conveying the entire story in a typical comic, and there exist a range of ways in which the information is distributed between words and image. Not only do comics offer an ideal medium for exploring these research questions, but in doing so we are able to use our findings to provide new insights into comics theory. There is a growing research field associated with understanding how we read and understand comics and theories are emerging that relate the manner in which images and words are used when creating comics to the reader's experience in terms of how they view and understand comics. Moreover, these ideas often drawn upon psychological phenomena and theory, yet have not been explored in within Psychological research or with the scientific method that underlies Experimental Psychology. At present comics theories lack empirical testing. Our work will be a first test of the assumptions that underpin comics theory and will therefore offer crucial new insights for this field and help shape the direction that this emerging field takes in the future and in particular will create a psychologically rigorous domain of research for this field.

文字和图像一起用来传达我们完成行为目标所需的广泛信息，包括对我们的安全至关重要的信息（安全标志），我们对世界的理解和互动（说明手册，地图，方向标志），我们做出的选择（广告）和娱乐（卡通，漫画）。在这些情况下，不仅文字和图像结合在一起，而且往往只有通过理解文字和图像的结合——它们共同传达的信息——我们才能理解被传达给我们的是什么。尽管文字-图像组合在我们周围的世界中无处不在，我们也依赖于对它们所传达的信息的理解，但我们对如何看待和理解这类刺激知之甚少。我们的大部分知识仅限于我们如何看待和理解单词、文本段落或图像。从这些先前的方法中，我们已经发展出了详细的理解，在许多方面，我们检查图像和文字的方式是非常不同的：不同的因素影响着我们所看的东西，甚至可能是我们在阅读时与在观看图像时不同的潜在处理风格和认知过程。这些对观看和处理文字和图像的截然不同的描述，引发了我们如何收集和理解包含文字和图像的刺激的问题。拟议的工作旨在为这一目前欠发达的研究领域提供关键见解。为了更好地理解我们如何检查和理解单词和图像组合的刺激，必须解决三个关键问题：(1)我们使用什么策略从单词和图像组合中收集信息？这些策略是否会随着单词和图像之间的关系而变化？也就是说，当文字传达的信息量与图像相比发生变化时，我们对这些刺激的看法会发生变化吗？(2)我们如何理解和回应词-象组合中的空间？文字之间的空间与图像中物体之间的空间传达的信息非常不同。(3)在直接看单词-图像组合之前，我们处理了多少信息？我们在周边视觉中处理文字和图像的某些方面，但我们这样做的程度是有争议的。漫画为解决这三个问题提供了一个理想的媒介，因为它们使用文字-图像组合向读者传达一个故事，通过几个面板和页面，读者必须一起理解所传达的信息。重要的是，漫画中文字和图像的结合是向读者传达故事的方式，在典型的漫画中，文字和图像都不能单独传达整个故事，信息在文字和图像之间的分布方式是多种多样的。漫画不仅为探索这些研究问题提供了理想的媒介，而且在这样做的过程中，我们能够利用我们的发现为漫画理论提供新的见解。有一个越来越多的研究领域与理解我们如何阅读和理解漫画有关，并且理论正在兴起，这些理论将漫画创作时使用的图像和文字的方式与读者如何看待和理解漫画的体验联系起来。此外，这些想法往往来自心理学现象和理论，但尚未在心理学研究中或实验心理学基础的科学方法中进行探索。目前漫画理论缺乏实证检验。我们的工作将是对支撑漫画理论的假设的第一次测试，因此将为这一领域提供至关重要的新见解，并帮助塑造这一新兴领域未来的发展方向，特别是将为这一领域创造一个心理学上严谨的研究领域。