CAREER: Understanding Vision and Natural Motion Statistics Through the Lens of Prediction

输入到大脑的视觉信息甚至在信号离开眼睛之前就已经被转换了，这些计算产生了对自然视觉世界结构的有效表示。PI以前的工作表明，这种处理可以包括重新包装信息以实现最佳预测。这表明了一种新的神经编码方法。虽然以前的许多研究都试图描述过去的刺激引起了随后的反应，这项工作要求这些反应预测未来的刺激。该项目将根据物体在外部世界中的移动方式得出最佳的预测结果，并量化大脑与最佳状态的接近程度。通过预测的透镜来观察大脑，可以发展出一种神经编码和计算的原理，可以将大脑区域从视网膜连接到更高的视觉区域。该计划的一个组成部分涉及测量和量化自然运动的预测成分。在此过程中，将创建一个自然运动的公共数据库，这将成为神经科学和计算机视觉社区的持久工具。一个相关的教育项目每年将把100多名当地中学生带到校园进行神经科学实验，并将向一大群研究生灌输科学教学的奖励和责任。这里提出的研究以各种方式探索视觉系统中的预测：通过计算复杂运动的预测编码的效率界限，通过开发定量方法来测试神经数据集中的这些界限，通过测量自然场景中的运动统计，通过描述大脑是如何机械地实现这一功能的。关于大脑如何执行最佳预测计算的假设可能受到自然视觉世界中可预测事件结构的约束。为了测量这些统计数据，将通过对自然场景进行高速、高像素深度的记录来构建一个新的自然电影数据库。通过量化这些数据中的运动，该项目将产生自然运动的统计和生成模型，这将为我们对自然世界的理解提供信息，并提供一种紧凑的方式来重现自然运动。这些刺激将用于测试神经系统是否最佳地编码与预测相关的信息。这项工作还将测试大脑中最佳预测的基础是什么自适应和其他非线性处理步骤。

The visual input to the brain is transformed even before signals leave the eye, and these computations produce an efficient representation of the structure of the natural visual world. Previous work by the PI has shown that this processing can include repackaging of information for optimal prediction. This suggests a new approach to neural encoding. While many previous studies have sought to characterize what stimuli in the past gave rise to a subsequent response, this work asks what future stimuli those responses predict. The proposed project will derive the best possible predictor given the way objects move in the outside world and quantify how close the brain gets to this optimum. Viewing the brain through the lens of prediction develops a principle of neural coding and computation that can bridge brain regions, from the retina to higher visual areas. A component of this plan involves measuring and quantifying the predictive components of natural motion. In doing so, a public database of natural motion will be created that will be a lasting tool for the neuroscience and computer vision communities. An associated educational program will bring over 100 local middle school children to campus each year for hands-on neuroscience experiments, and will instill in a large group of graduate students the rewards and responsibilities of science teaching.The research proposed here explores prediction in the visual system in a variety of ways: by computing efficiency bounds on the predictive encoding of complex motion, by developing quantitative methods to test these bounds in neural datasets, by measuring the statistics of motion in natural scenes, and by describing how, mechanistically, the brain achieves this performance. Hypotheses about how the brain performs optimal predictive computations may be constrained by the structure of predictable events in the natural visual world. To measure these statistics, a new natural movie database will be constructed by making high-speed, high-pixel-depth recordings of natural scenes. By quantifying motion in these data, this project will yield statistical and generative models of natural motion that will inform our understanding of the natural world and provide a compact way to recapitulate natural motion in silico. These stimuli will be used to test whether neural systems optimally encode information relevant for prediction. The work will also test what adaptive and otherwise non-linear processing steps underlie optimal prediction in the brain.