CAREER: Understanding Vision and Natural Motion Statistics Through the Lens of Prediction

甚至在信号离开眼睛之前，大脑的视觉输入就被转换了，这些计算产生了自然视觉世界结构的有效表示。PI先前的工作表明，这种处理可以包括对信息的重新包装，以实现最佳预测。这提出了一种新的神经编码方法。虽然以前的许多研究都试图描述过去的刺激会引起随后的反应，但这项工作询问了这些反应预测未来的刺激。该计划将根据物体在外界的运动方式，推导出可能的最佳预测器，并量化大脑离这一最佳状态有多近。通过预测的视角来观察大脑，会发展出一种神经编码和计算的原理，这种原理可以连接大脑区域，从视网膜到更高的视觉区域。该计划的一个组成部分包括测量和量化自然运动的预测成分。在此过程中，自然运动的公共数据库将被创建，这将成为神经科学和计算机视觉社区的持久工具。一个相关的教育项目每年将吸引100多名当地中学生到校园进行动手神经科学实验，并将向一大群研究生灌输科学教学的回报和责任。这里提出的研究以多种方式探索视觉系统中的预测：通过计算复杂运动预测编码的效率界限，通过开发定量方法在神经数据集中测试这些界限，通过测量自然场景中的运动统计，以及通过描述大脑如何机械地实现这种性能。关于大脑如何进行最佳预测计算的假设可能受到自然视觉世界中可预测事件结构的限制。为了测量这些统计数据，将通过对自然场景进行高速、高像素深度的记录来构建一个新的自然电影数据库。通过量化这些数据中的运动，该项目将产生自然运动的统计和生成模型，这将告知我们对自然世界的理解，并提供一种在计算机上概括自然运动的紧凑方法。这些刺激将被用来测试神经系统是否对与预测相关的信息进行最佳编码。这项工作还将测试哪些自适应和非线性处理步骤是大脑最佳预测的基础。

The visual input to the brain is transformed even before signals leave the eye, and these computations produce an efficient representation of the structure of the natural visual world. Previous work by the PI has shown that this processing can include repackaging of information for optimal prediction. This suggests a new approach to neural encoding. While many previous studies have sought to characterize what stimuli in the past gave rise to a subsequent response, this work asks what future stimuli those responses predict. The proposed project will derive the best possible predictor given the way objects move in the outside world and quantify how close the brain gets to this optimum. Viewing the brain through the lens of prediction develops a principle of neural coding and computation that can bridge brain regions, from the retina to higher visual areas. A component of this plan involves measuring and quantifying the predictive components of natural motion. In doing so, a public database of natural motion will be created that will be a lasting tool for the neuroscience and computer vision communities. An associated educational program will bring over 100 local middle school children to campus each year for hands-on neuroscience experiments, and will instill in a large group of graduate students the rewards and responsibilities of science teaching.The research proposed here explores prediction in the visual system in a variety of ways: by computing efficiency bounds on the predictive encoding of complex motion, by developing quantitative methods to test these bounds in neural datasets, by measuring the statistics of motion in natural scenes, and by describing how, mechanistically, the brain achieves this performance. Hypotheses about how the brain performs optimal predictive computations may be constrained by the structure of predictable events in the natural visual world. To measure these statistics, a new natural movie database will be constructed by making high-speed, high-pixel-depth recordings of natural scenes. By quantifying motion in these data, this project will yield statistical and generative models of natural motion that will inform our understanding of the natural world and provide a compact way to recapitulate natural motion in silico. These stimuli will be used to test whether neural systems optimally encode information relevant for prediction. The work will also test what adaptive and otherwise non-linear processing steps underlie optimal prediction in the brain.