Cortical Motion Coding and Gaze Control in Natural Vision

人眼向大脑发送信息的速度估计为每秒10兆比特，大致相当于以太网连接的速度。在行为相关的时间尺度上处理如此大带宽的视觉信息流需要大脑有效地从视觉信号中提取和表示信息，即以最少的时间、硬件和能量成本表示最多的信息。从本质上讲，大脑需要压缩视觉流，就像软件压缩电影的数字表示一样。这种编码增强可能是因为大脑已经进化出了编码策略，这些策略特别考虑了这样一个事实，即由于物体和眼睛的运动，眼睛的视觉输入可能在空间和时间上是相关的。因此，从眼睛和视网膜到大脑的视觉信号可能是相当可预测的。当前感觉-运动系统研究中的一个主要问题是，大脑在多大程度上利用预测来补偿处理信息所需的有限时间，即使视觉场景可能在此期间发生变化。该建议侧重于自然运动视频的视觉运动和凝视行为的神经表示，并使用一种新的视频游戏环境来简化凝视的分析。该项目还将创建一个公开的自然凝视记录数据库，分析自然视网膜图像运动的统计数据，表征皮层神经元中自然相关运动刺激的表示，并阐明凝视控制的策略。这个数据库将有益于神经科学、计算机视觉、媒体设计和其他领域。实验方法将非人类灵长类动物的皮层生理学与人类和猴子的高分辨率眼动记录相结合。PI建议使用自然运动场景的高分辨率视频作为视觉刺激，同时记录运动敏感视觉皮层中的神经活动。 通过仔细地将电影降级，使其越来越不自然，并测量对神经反应的影响，实验将确定移动视觉场景的哪些特征最精确地表示。 第二组实验将研究视觉场景和眼球运动之间的相互作用。PI将开发一种创新的Pong式视频游戏，积极吸引观众并创建共同的观看目的（得分），同时简化感兴趣目标的识别以帮助分析，从而控制观众的认知状态。这项工作的跨学科性质将为本科生和研究生提供从数学和物理学到神经生物学的培训机会，并为具有生物学背景的学生提供计算分析技能。

The human eye sends information to the brain at an estimated rate of approximately 10 megabits per second, roughly the speed of an ethernet connection. Processing such a large bandwidth stream of visual information on behaviorally relevant time scales requires the brain to extract and represent information from visual signals efficiently, i.e. represent the most information for the least cost in time, hardware and energy. In essence, the brain needs to compress the visual stream in much the same way that software compresses the digital representation of a movie. This coding enhancement might arise because the brain has evolved coding strategies that specifically account for the fact that because of both object and eye movements, the visual input to the eye may be correlated in space and time. As a result, the visual signals to the brain from the eye and retina may be quite predictable. One of the primary questions in current sensory-motor systems research is to what extent the brain utilizes prediction to compensate for the fact that it takes a finite amount of time to process information even though the visual scene might change in the interim. This proposal focuses on neural representation of visual motion and gaze behavior for natural motion videos and uses a novel video game environment to simplify the analysis of gaze. The project will also create a publicly available database of natural gaze recordings, analyze the statistics of natural retinal image motion, characterize the representation of naturally correlated motion stimuli in cortical neurons, and to articulate the strategy underlying gaze control. This database will benefit neuroscience, computer vision, media design, and other fields.The experimental approach combines cortical physiology in non-human primates with high-resolution eye movement recording in both humans and monkeys. The PI proposes to use high-resolution videos of natural moving scenes as visual stimuli while recording neural activity in motion-sensitive visual cortex. By carefully degrading the movies to make them increasingly less natural and measuring the impact on neural responses, the experiments will determine what features of the moving visual scene are represented most precisely. A second set of experiments will study the interactions between the visual scene and eye movements. The PI will develop an innovative Pong-like video game that actively engages the viewers and creates a common viewing purpose (scoring points) while simplifying the identification of the target of interest to aid analysis, thereby controlling the cognitive state of the viewer. The interdisciplinary nature of the work will provide training opportunities for undergraduate and graduate students crossing over from mathematics and physics to neurobiology, and for students with a biology background to gain skills in computational analysis.