Computational foundations of active visual sensing

摘要 视觉是一个主动的过程：我们移动我们的头和眼睛来探索感官世界。这是特别 在静止的观察者提供有限信息的情况下，例如在寻找物体时， 这在复杂的自然场景中很常见。然而，我们对积极的理解 视觉是有限的，由于实验和理论的挑战，包括自由研究视觉的困难， 移动的动物和缺乏正式的理论框架，整合视觉表现与行动。 在这个团队项目中，我们将联合收割机结合视觉神经科学，行为，机器学习和理论方面的专业知识， 以确定主动感知的行为、神经和计算基础。我们的做法是根据 在有限理性控制（BRC）的一个新的理论框架，和一个行为任务，小鼠执行 在存在遮挡和图像损坏的情况下的对象识别任务。为了启用主动感知，刺激 基于动物的视角在增强现实中实时呈现。在我们的第一个目标，我们将 基于BRC中的受限视觉表示，开发主动感知模型。第二个目标，我们 测量任务期间的行为表现（正确/不正确的反应和全身运动）， 在第三个目标中，我们将测量任务期间视觉皮层区域的神经活动。两个目标 2和3，我们将我们的模型拟合到相应的行为和神经数据，然后进行因果检验 我们的模型，通过提出新的刺激，预测引起特定的反应，从模型。所有这些 目标将提供对鼠标主动视觉的基本理解，这将支持随后的U19 建议利用遗传工具来研究潜在的本地和远程神经回路。

Abstract Vision is an active process: we move our head and eyes to explore the sensory world. This is particularly important in situations where a stationary view provides limited information, such as when looking for an object that is occluded or obscured, which is common in complex natural scenes. However, our understanding of active vision is limited due to experimental and theoretical challenges, including the difficulty of studying vision in freely moving animals and the lack of formal theoretical frameworks that integrate visual representations with actions. In this team project, we will combine expertise in visual neuroscience, behavior, machine learning, and theory, to determine the behavioral, neural, and computational underpinnings of active sensing. Our approach is based on a new theoretical framework of Bounded Rational Control (BRC), and a behavioral task in which mice perform an object recognition task in the presence of occlusion and image corruptions. To enable active sensing, stimuli in the task are rendered real-time in augmented reality based on the animal's viewpoint. In our first aim, we will develop models of active sensing based on constrained visual representations in BRC. In the second aim, we measure behavioral performance (both correct/incorrect responses and full-body movements) during the task, and in the third aim we will measure neural activity across visual cortical areas during the task. For both Aims 2 and 3, we will fit our models to the corresponding behavioral and neural data, and then perform causal tests of our models by presenting novel stimuli predicted to elicit specific responses from the model. Together, these aims will provide a foundational understanding of active vision in the mouse that will support a subsequent U19 proposal taking advantage of genetic tools to investigate the underlying local and long-range neural circuits.