DynaVision - Representational Dynamics in Vision

脑科学的核心挑战之一是理解我们从周围世界丰富的视觉信息中提取意义的能力背后的皮层机制。近年来，从机器学习中借鉴的多变量模式识别技术的应用，使人们对视觉信息在皮层网络中的表现和转换方式有了前所未有的了解。为了评估皮质表征的细粒度结构，大多数先前的人类研究依赖于功能性磁共振成像，其提供相对较高的空间分辨率，但时间分辨率较差。因此，我们对视觉表征的大部分知识都依赖于大脑皮层活动的时间平均值，而忽略了在较短时间尺度上运作的编码机制，在实验取得进展的同时，计算神经科学和计算机视觉走了一条互补的道路，转而关注人工视觉系统。受神经科学的启发，最近的深度神经网络模型（DNN）在基本的对象识别任务上取得了巨大的进步，达到了前所未有的精度，接近人类的性能水平。有趣的是，最成功的前馈模型的内部表征与皮层表征有着惊人的相似之处。然而，这些比较再次基于时间平均数据。因此，目前还不清楚DNN可以在多大程度上解释动态变化的皮层信号。视觉表征的实验和计算研究都集中在空间活动模式和前馈计算上，使得视觉处理的时间动态性知之甚少。在这里，我提出了一个研究计划，以促进我们的知识的潜在快速表征变化的大脑。我将开发新的多变量分析和统计推断技术，可以揭示皮层编码阶段的顺序及其潜在的计算机制。将这些方法应用于以高时间分辨率记录的脑磁图数据，我将进一步研究大脑是否使用频率复用来同时保持和传输不同的表示。作为对这些分析的补充，我将使用最先进的DNN来构建动态复杂性不断增加的模型，以确定人类皮层视觉处理的计算机制和动态是否最好由前馈或递归深度神经网络模型来解释。总之，拟议的研究项目将为大脑中视觉表征动态的机制提供重要的见解，基于新颖的计算模型和数据分析技术。这套新颖的工具将进一步使整个系统神经科学的研究人员能够探索生物大脑的表征动力学，并测试计算模型和实验数据之间的一致性。

One of the central challenges of brain science is to understand the cortical mechanisms that underlie our ability to extract meaning from the rich visual information in the world around us. In recent years, the application of multivariate pattern recognition techniques, borrowed from machine learning, has led to unprecedented insights into how visual information is represented and transformed in the cortical network. To assess the fine-grained structures of cortical representations, most previous human studies rely on functional magnetic resonance imaging, which offers relatively high spatial, but poor temporal resolution. Much of our knowledge about visual representations, therefore, rests on temporal averages of cortical activity, neglecting encoding schemes that operate on shorter time scales.In parallel to experimental advances, computational neuroscience and computer vision have taken a complementary route, focusing instead on artificial vision systems. Inspired by neuroscience, recent deep neural network models (DNNs) enabled great strides at basic object recognition tasks, reaching unprecedented accuracies that approach human performance levels. Interestingly, the internal representations of the most successful feedforward models exhibit striking similarities to cortical representations. These comparisons are, however, again based on temporally averaged data. It therefore remains unclear in how far DNNs can account for dynamically changing cortical signals.Both, experimental and computational studies of visual representations have focussed on spatial activity patterns and feedforward computations, leaving the temporal dynamics of visual processing poorly understood. Here I propose a research program to advance our knowledge of the potentially rapid representational changes in the brain. I will develop novel multivariate analysis and statistical inference techniques, which can reveal the sequence of cortical encoding stages and their underlying computational mechanisms. Applying these methods to magnetoencephalography data, recorded at high temporal resolution, I will furthermore investigate whether the brain uses multiplexing by frequency to concurrently maintain and transfer different representations. Complementing these analyses, I will use state-of-the-art DNNs to build models of increasing dynamic complexity to determine whether the computational mechanisms and dynamics of human cortical visual processing are best accounted for by a feedforward or a recurrent deep neural network model.In summary, the proposed research project will provide important insights into the mechanisms underlying the dynamics of visual representation in the brain, based on novel computational models and data analysis techniques. This novel set of tools will furthermore enable researchers throughout systems neuroscience to probe the representational dynamics of biological brains and to test the agreement between computational models and experimental data.