CAREER: Mechanisms enabling the flexible expression of visual concepts

以视觉形式表达思想的技术在人类历史上一直至关重要。从古代洞穴壁画到现代数字图形，这些技术是我们一些最重要的发明的核心，包括艺术，写作和数学。尽管这些技术很重要，但人们对人类大脑如何以如此不同的方式使用它们知之甚少。也许这些技术中最基本和最通用的是绘图，它可以用于在许多抽象层次上传达有关视觉世界的信息，从逼真的插图到简化的图表。该项目将利用大型数据集和先进的数据分析技术，对人们使用绘图以不同方式传达视觉概念时所涉及的心理过程进行严格的理解。该项目的结果将促进我们对为什么人们更喜欢在某些情况下使用某些类型的图像而不是其他图像的理解，并对如何为各种应用设计有效的可视化产生影响，包括STEM教育和研究。这个项目的重点是如何抽象的想法可以在清晰和可访问的方式进行沟通的问题延伸到其教育计划。该计划包括两个倡议，以发展包容性的学习经验，促进K-12和本科生的计算素养，这些学生在历史上面临着系统性的障碍，这种培训。该项目的目标1旨在解决一个经典的争论，即绘画是否通过模仿世界上的物体来获得其意义（即，是图像状的）或由离散的符号表达式组成（即，是语言类的）。拟议的实验将评估的假设，即图纸既不是纯粹的图像，也不是纯粹的语言一样，但可以有很大的不同，这取决于插画家可以看到什么，他们知道什么，他们希望沟通的信息。为了验证这一假设，拟议的分析将采用众包和计算机视觉技术来衡量不同的绘画保留感知信息和/或组织成离散符号单元的程度。例如，一些绘图可能包含丰富的视觉细节，这些细节难以用文字概括，而其他绘图可能完全由更简单的标记组成，这些标记可以很容易地用文字描述。目标2将研究人们随着时间的推移想出新的视觉交流方式的过程。拟议的实验将评估一个假设，即最初类似于一个具体物体的绘画往往会变得越来越抽象和符号化，当人们反复沟通时，反映了沟通者之间的共同目标和知识。拟议的分析将衡量所产生的图纸的一致性和可变性，提供定量的洞察力的因素，影响新的符号系统的通信的发展。目标3旨在更好地调整科学培训本科心理学学生接受现代科学实践的心理学的现实。具体来说，它将开放科学最佳实践，探索性数据可视化和基于模型的数据分析的教学整合到加州圣地亚哥大学心理学的入门统计课程中，以及合作的最终研究项目，以帮助学生综合他们所学到的知识并磨练他们的沟通技巧。Aim 4致力于通过与当地K-12学校合作，在历史上服务不足的社区扩大有关人工智能（AI）的通识教育，以开发学习体验，说明AI与学生日常生活的相关性，以及AI如何与其他学科交叉，包括艺术，心理学，医学，该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Technologies for expressing ideas in visual form have been critically important throughout human history. From ancient cave paintings to modern digital graphics, such technologies lie at the heart of some of our most significant inventions, including art, writing, and mathematics. Despite the importance of such technologies, little is known about how the human mind is capable of using them in such varied ways. Perhaps the most basic and versatile of these technologies is drawing, which can be used to convey information about the visual world at many levels of abstraction, ranging from realistic illustrations to simplified diagrams. This project will harness large datasets and advanced data analysis techniques to develop a rigorous understanding of the mental processes involved when people use drawings to communicate visual concepts in different ways. Results from this project will advance our understanding of why people prefer to use certain kinds of images in some contexts and not others, with implications for how to design effective visualizations for a variety of applications, including STEM education and research. This project’s focus on the problem of how abstract ideas can be communicated in clear and accessible ways extends to its education plan. This plan encompasses two initiatives to develop inclusive learning experiences that promote computational literacy among K-12 and undergraduate students that have historically faced systemic obstacles to this training.Aim 1 of this project seeks to resolve a classic debate concerning whether drawings derive their meaning by resembling objects in the world (i.e., are image-like) or by being composed of discrete symbolic expressions (i.e., are language-like). The proposed experiments will evaluate the hypothesis that drawings are neither purely image-like nor purely language-like, but can vary strongly depending on what the illustrator can see, what they know, and what information they wish to communicate. To test this hypothesis, the proposed analyses will employ crowdsourcing and computer-vision techniques to measure the degree to which different drawings preserve perceptual information and/or are organized into discrete symbolic units. For example, some drawings may contain rich visual details that resist summarization in words, while other drawings may be entirely composed of simpler marks that can be readily described using words. Aim 2 will investigate the process by which people come up with new ways to visually communicate with each other over time. The proposed experiments will evaluate the hypothesis that drawings that initially resemble a concrete object tend to become increasingly abstract and symbol-like when people repeatedly communicate about it, reflecting shared goals and knowledge between communicators. The proposed analyses will measure consistency and variability in the resulting drawings, providing quantitative insight into the factors that affect the development of new symbolic systems for communication. Aim 3 seeks to better align the scientific training undergraduate psychology students receive to the reality of modern scientific practice in psychology. Specifically, it will integrate teaching of open-science best practices, exploratory data visualization, and model-based data analysis into the introductory statistics curriculum in Psychology at University of California San Diego, as well as collaborative final research projects to help students synthesize what they have learned and hone their communication skills. Aim 4 strives to broaden access to general education about artificial intelligence (AI) in historically underserved communities by partnering with local K-12 schools to develop learning experiences that illustrate the relevance of AI to students’ everyday lives, as well as how AI intersects with other disciplines, including the arts, psychology, medicine, and law.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.