CAREER: Understanding Vision and Natural Motion Statistics Through the Lens of Prediction

大脑的视觉输入甚至在信号离开眼睛之前就被转换了，这些计算产生了自然视觉世界结构的有效表示。PI以前的工作已经表明，这种处理可以包括对信息进行重新打包，以实现最佳预测。这提出了一种新的神经编码方法。虽然许多以前的研究试图描述过去的刺激会引起后续反应的特征，但这项工作询问的是这些反应预示着未来的刺激是什么。这项拟议的项目将根据物体在外部世界中的运动方式得出可能的最佳预测值，并量化大脑与该最佳预测值的接近程度。通过预测的镜头来观察大脑，发展了一种神经编码和计算的原理，可以连接大脑区域，从视网膜到高级视觉区域。该计划的一个组成部分包括测量和量化自然运动的预测成分。在这样做的过程中，将创建一个关于自然运动的公共数据库，它将成为神经科学和计算机视觉社区的持久工具。一个相关的教育项目每年将吸引100多名当地中学生到校园进行实践神经科学实验，并将向一大批研究生灌输科学教学的回报和责任。这里提出的研究以多种方式探索视觉系统的预测：通过计算复杂运动预测编码的效率界限，通过开发在神经数据集中测试这些界限的量化方法，通过测量自然场景中的运动统计数据，通过描述大脑如何机械地实现这一性能。关于大脑如何执行最佳预测计算的假设可能会受到自然视觉世界中可预测事件结构的限制。为了衡量这些统计数据，将通过对自然场景进行高速、高像素深度的记录来构建一个新的自然电影数据库。通过量化这些数据中的运动，这个项目将产生自然运动的统计和生成模型，这些模型将告诉我们对自然世界的理解，并提供一种简洁的方法来概括硅胶中的自然运动。这些刺激将被用来测试神经系统是否以最佳方式编码与预测相关的信息。这项工作还将测试大脑中哪些自适应和其他非线性处理步骤是最佳预测的基础。

The visual input to the brain is transformed even before signals leave the eye, and these computations produce an efficient representation of the structure of the natural visual world. Previous work by the PI has shown that this processing can include repackaging of information for optimal prediction. This suggests a new approach to neural encoding. While many previous studies have sought to characterize what stimuli in the past gave rise to a subsequent response, this work asks what future stimuli those responses predict. The proposed project will derive the best possible predictor given the way objects move in the outside world and quantify how close the brain gets to this optimum. Viewing the brain through the lens of prediction develops a principle of neural coding and computation that can bridge brain regions, from the retina to higher visual areas. A component of this plan involves measuring and quantifying the predictive components of natural motion. In doing so, a public database of natural motion will be created that will be a lasting tool for the neuroscience and computer vision communities. An associated educational program will bring over 100 local middle school children to campus each year for hands-on neuroscience experiments, and will instill in a large group of graduate students the rewards and responsibilities of science teaching.The research proposed here explores prediction in the visual system in a variety of ways: by computing efficiency bounds on the predictive encoding of complex motion, by developing quantitative methods to test these bounds in neural datasets, by measuring the statistics of motion in natural scenes, and by describing how, mechanistically, the brain achieves this performance. Hypotheses about how the brain performs optimal predictive computations may be constrained by the structure of predictable events in the natural visual world. To measure these statistics, a new natural movie database will be constructed by making high-speed, high-pixel-depth recordings of natural scenes. By quantifying motion in these data, this project will yield statistical and generative models of natural motion that will inform our understanding of the natural world and provide a compact way to recapitulate natural motion in silico. These stimuli will be used to test whether neural systems optimally encode information relevant for prediction. The work will also test what adaptive and otherwise non-linear processing steps underlie optimal prediction in the brain.