CompCog: Collaborative Research: Learning Visuospatial Reasoning Skills from Experience

该项目使用人工智能（AI）的方法来更好地了解人们如何学习视觉空间推理技能，如心理旋转，这是发展强大的数学和科学能力的关键因素。 特别是，该项目提出了一种新的方法来量化不同视觉体验中包含的学习价值，使用可穿戴相机结合新的AI系统，从视频示例中学习视觉空间推理技能。 该项目的结果不仅将推动人工智能的发展，还将使研究人员能够衡量不同的现实世界视觉体验在帮助人们学习视觉空间推理技能方面的价值。 例如，某些类型的物体游戏活动可能对帮助儿童学习某些视觉空间推理技能特别有价值。 最终，这种新的测量方法可以用来识别儿童视觉空间推理困难的早期迹象，也可以帮助设计新的视觉空间训练干预措施，以促进儿童的早期数学和科学发展。该项目旨在回答的核心科学问题是：视觉空间推理技能是如何从第一人称视觉体验中学习的？ 这个问题将通过一个新的人工智能系统的计算实验来回答--心理图像引擎（MIME）--它从视频示例中学习视觉空间推理技能，如心理旋转。 训练数据将包括来自两种不同环境的第一人称可穿戴摄像头视频，这两种环境对人类学习都很重要：婴儿的非结构化物体操作和为儿童设计的视觉空间训练干预。 MIME人工智能系统的实验结果将通过帮助解释如何从视觉体验中学习视觉空间推理技能，特别是不同类型的视觉体验如何以不同的方式影响一个人的学习成果的质量，来推动人工智能和人类学习科学的发展。

This project uses methods from artificial intelligence (AI) to better understand how people learn visuospatial reasoning skills like mental rotation, which are a critical ingredient in the development of strong math and science abilities. In particular, this project proposes a new approach to quantify the learning value contained in different visual experiences, using wearable cameras combined with a new AI system that learns visuospatial reasoning skills from video examples. Results from this project will not only advance the state of the art in AI but also will enable researchers to measure how valuable different real-world visual experiences are in helping people to learn visuospatial reasoning skills. For example, certain types of object play activities might be particularly valuable for helping a child to learn certain visuospatial reasoning skills. Ultimately, this new measurement approach could be used to identify early signs of visuospatial reasoning difficulties in children and could also help in the design of new visuospatial training interventions to boost children's early math and science development.The core scientific question that this project aims to answer is: How are visuospatial reasoning skills learned from first-person visual experiences? This question will be answered through computational experiments with a new AI system---the Mental Imagery Engine (MIME)---that learns visuospatial reasoning skills, like mental rotation, from video examples. Training data will include first-person, wearable-camera videos from two different settings that are both important for human learning: unstructured object manipulation by infants and visuospatial training interventions designed for children. Results from experiments with the MIME AI system will advance the state of the art in both AI and the science of human learning by helping to explain how visuospatial reasoning skills can be learned from visual experiences, and, in particular, how having different kinds of visual experiences can affect the quality of a person's learning outcomes in different ways.