Fast and Flexible Conjunction Coding in Biological and Artificial Vision

生物和人工视觉中快速灵活的联合编码

• 批准号: 10389898
• 负责人: JohnMark Edward Taylor
• 金额: $ 6.68万
• 依托单位: COLUMBIA UNIV NEW YORK MORNINGSIDE
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-04-16 至 2025-04-15
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10389898
• 关键词: Anatomy; Architecture; Artificial Intelligence; Attention; Automobile Driving; Behavior; Biological; Brain; Code; Cognitive; Cognitive Science; Color; Complex; Computer Models; Computer Vision Systems; Crowding; Data Set; Development; Doctor of Philosophy; Engineering; Environment Design; Exhibits; Feedback; Functional Magnetic Resonance Imaging; Future; Glass; Homologous Gene; Human; Image; Industrialization; Influentials; Lateral; Link; Literature; Mentorship; Methodology; Methods; Modeling; Modernization; Names; Natural experiment; Neurologic Deficit; Neurons; Neurosciences; Patients; Population; Prevalence; Psychophysics; Recurrence; Research; Safety; Shapes; Stimulus; Stream; Sum; Syndrome; Techniques; Technology; Testing; Toy; Training; Vision; Visual; Visual Agnosias; Visual system structure; Work; artificial neural network; base; career; cognitive neuroscience; computational neuroscience; deep learning; deep neural network; experience; experimental study; flexibility; human model; improved; neuroimaging; neurophysiology; novel; object recognition; open source; operation; presynaptic; relating to nervous system; response; synergism; theories; vision science; visual search

Project Summary How does the human brain encode visual feature conjunctions? Two influential, yet disparate, research traditions have proposed two different mechanisms. Research in neurophysiology and computer vision has explored static conjunction coding, where feature conjunctions are automatically extracted via a feedforward hierarchy. While efficient, this mechanism may be limited to encoding conjunctions enabled by the hierarchy's learned connectivity. By contrast, research in cognitive psychology has explored dynamic conjunction coding, which sequentially encodes task-relevant conjunctions via attentional selection. While slow, this mechanism is flexible and capable of encoding any feature conjunction. Despite evidence for both mechanisms, their interplay remains unclear. In this project, we leverage advances in deep learning and open neuroimaging datasets to understand how these two mechanisms interact in the human brain, yielding the best of both worlds: fast, but flexible conjunction coding. Through the three complementary Specific Aims, we advance our understanding of these fundamental issues in vision science, and on a practical level, develop approaches that can be used to improve computer vision, aiding the development of useful technologies like autonomous vehicles. In the course of this project, I will master modern approaches in deep learning and computational neuroscience through the mentorship of my sponsor Dr. Kriegeskorte and my co-sponsor Dr. Fusi, equipping me for a career leading a lab that bridges cognitive science, neuroscience, and artificial intelligence. Hypotheses: The human brain implements static conjunction coding via neural populations with “and-like” tuning to feature combinations that emerges via feedforward convergence of neurons tuned to single features, while dy- namic conjunction coding requires recurrent connections. Static conjunction coding can rapidly encode familiar, but not unfamiliar conjunctions, while dynamic conjunction coding can encode any conjunction, but more slowly. Aim 1: We use a massive open-source fMRI dataset to chart the prevalence of static conjunction coding through- out the human visual system, using a method we developed in preliminary analyses. Aim 2: We apply “synthetic neurophysiology” to feedforward artificial neural networks to understand how static conjunction coding emerges in a feedforward hierarchy, beginning by characterizing conjunction-tuned units iden- tified in preliminary analyses, followed by testing influential models for how this occurs in biological vision. Aim 3: We test whether feedforward artificial neural networks exhibit similar limitations on a visual search task compared to known results for human feedforward vision, followed by testing whether introducing recurrent con- nections to networks can overcome these limitations.

项目摘要 人脑是如何编码视觉特征连词的？两个人在fl潜在的，但截然不同的研究传统 提出了两种不同的机制。神经生理学和计算机视觉的研究探索了静态 合取编码，其中特征合取通过前馈层次自动提取。而当 如fi所知，此机制可能仅限于编码由层次结构的学习连通性启用的合取。 相比之下，认知心理学的研究探索了动态连词编码，这是依次进行的 通过注意选择对任务相关的连词进行编码。虽然速度很慢，但这种机制是fl可扩展的，并且有能力 对任何特征连接进行编码。尽管有证据表明这两种机制都存在，但它们之间的相互作用仍不清楚。在……里面 在这个项目中，我们利用深度学习方面的进展和开放的神经成像数据集来了解这些 两种机制在人脑中相互作用，产生了两全其美的结果：快速但fl灵活的连接词编码。 通过三个相辅相成的规范fic目标，我们促进了我们对这些基本问题的理解 视觉科学，并在实践层面上，开发可用于改善计算机视觉的方法，帮助 自动驾驶汽车等有用技术的开发。在这个项目的过程中，我将掌握 深度学习和计算神经科学的现代方法通过我的赞助人Dr. Kriegeskorte和我的联合赞助人Fusi博士，为我领导一个连接认知科学的实验室的职业生涯做好了准备， 神经科学，和艺术fi社会智力。 假设：人脑通过神经种群实现静态连接编码，并进行“AND-like”调整 通过调谐到单个特征的神经元的前馈收敛而出现的特征组合，而不是 NAMIC连接编码需要循环连接。静态联合编码可以快速地编码熟悉的、 但不是不熟悉的连词，而动态连词编码可以对任何连词进行编码，但速度较慢。 目标1：我们使用一个大规模的开源fMRI数据集来绘制静态连接编码的流行度图表，通过- 使用我们在初步分析中开发的方法，对人类视觉系统进行了分析。 目的2：我们将“合成神经生理学”应用于前馈神经网络，以了解如何静态。 合取编码出现在前馈层次结构中，首先表征合取调谐的单元。 fi进行了初步分析，然后在fl潜在模型中测试了这在生物视觉中是如何发生的。 目的3：我们测试了前馈神经网络在视觉搜索任务中是否表现出类似的限制。 与人类前馈视觉的已知结果进行比较，然后测试是否引入复发性屈光度。 连接到网络可以克服这些限制。