DynaVision - Representational Dynamics in Vision

脑科学的核心挑战之一是理解我们从周围世界丰富的视觉信息中提取意义的能力所依据的大脑皮层机制。近年来，借鉴机器学习的多变量模式识别技术的应用，使人们对视觉信息在大脑皮层网络中的表示和转换有了前所未有的深入了解。为了评估大脑皮质表征的细粒度结构，以前的大多数人类研究都依赖于功能磁共振成像，它提供了相对较高的空间分辨率，但时间分辨率较差。因此，我们关于视觉表征的大部分知识依赖于大脑皮质活动的时间平均，忽略了在较短时间尺度上运行的编码方案。在实验进步的同时，计算神经科学和计算机视觉采取了一条互补的路线，转而专注于人工视觉系统。在神经科学的启发下，最近的深度神经网络模型(DNN)使基本对象识别任务取得了巨大进展，达到了前所未有的精度，接近人类的表现水平。有趣的是，最成功的前馈模型的内部表征与大脑皮层表征有惊人的相似之处。然而，这些比较同样是基于时间平均数据。因此，目前尚不清楚DNN在多大程度上能够解释动态变化的皮层信号。视觉表征的实验和计算研究都集中在空间活动模式和前馈计算上，而视觉加工的时间动力学却知之甚少。在这里，我提出了一个研究计划，以促进我们对大脑中潜在的快速代表性变化的了解。我将开发新的多变量分析和统计推断技术，它可以揭示大脑皮质编码阶段的序列及其潜在的计算机制。将这些方法应用于以高时间分辨率记录的脑磁图数据，我将进一步调查大脑是否使用频率多路传输来同时保持和传输不同的表示。作为对这些分析的补充，我将使用最先进的DNN来建立动态复杂性不断增加的模型，以确定人类大脑皮层视觉处理的计算机制和动力学是由前馈模型还是递归深度神经网络模型来最好地解释的。综上所述，基于新的计算模型和数据分析技术，拟议的研究项目将为揭示大脑中视觉表征动力学的机制提供重要的见解。这套新的工具还将使整个系统神经科学的研究人员能够探索生物大脑的表征动力学，并测试计算模型和实验数据之间的一致性。

One of the central challenges of brain science is to understand the cortical mechanisms that underlie our ability to extract meaning from the rich visual information in the world around us. In recent years, the application of multivariate pattern recognition techniques, borrowed from machine learning, has led to unprecedented insights into how visual information is represented and transformed in the cortical network. To assess the fine-grained structures of cortical representations, most previous human studies rely on functional magnetic resonance imaging, which offers relatively high spatial, but poor temporal resolution. Much of our knowledge about visual representations, therefore, rests on temporal averages of cortical activity, neglecting encoding schemes that operate on shorter time scales.In parallel to experimental advances, computational neuroscience and computer vision have taken a complementary route, focusing instead on artificial vision systems. Inspired by neuroscience, recent deep neural network models (DNNs) enabled great strides at basic object recognition tasks, reaching unprecedented accuracies that approach human performance levels. Interestingly, the internal representations of the most successful feedforward models exhibit striking similarities to cortical representations. These comparisons are, however, again based on temporally averaged data. It therefore remains unclear in how far DNNs can account for dynamically changing cortical signals.Both, experimental and computational studies of visual representations have focussed on spatial activity patterns and feedforward computations, leaving the temporal dynamics of visual processing poorly understood. Here I propose a research program to advance our knowledge of the potentially rapid representational changes in the brain. I will develop novel multivariate analysis and statistical inference techniques, which can reveal the sequence of cortical encoding stages and their underlying computational mechanisms. Applying these methods to magnetoencephalography data, recorded at high temporal resolution, I will furthermore investigate whether the brain uses multiplexing by frequency to concurrently maintain and transfer different representations. Complementing these analyses, I will use state-of-the-art DNNs to build models of increasing dynamic complexity to determine whether the computational mechanisms and dynamics of human cortical visual processing are best accounted for by a feedforward or a recurrent deep neural network model.In summary, the proposed research project will provide important insights into the mechanisms underlying the dynamics of visual representation in the brain, based on novel computational models and data analysis techniques. This novel set of tools will furthermore enable researchers throughout systems neuroscience to probe the representational dynamics of biological brains and to test the agreement between computational models and experimental data.