Fast and Flexible Conjunction Coding in Biological and Artificial Vision

生物和人工视觉中快速灵活的联合编码

• 批准号: 10611323
• 负责人: JohnMark Edward Taylor
• 金额: $ 6.91万
• 依托单位: COLUMBIA UNIV NEW YORK MORNINGSIDE
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-04-16 至 2025-04-15
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10611323
• 关键词: Anatomy; Architecture; Artificial Intelligence; Attention; Automobile Driving; Behavior; Biological; Brain; Code; Cognitive; Cognitive Science; Color; Complex; Computer Models; Computer Vision Systems; Crowding; Data Set; Development; Disparate; Doctor of Philosophy; Endowment; Engineering; Environment Design; Exhibits; Feedback; Functional Magnetic Resonance Imaging; Future; Glass; Homologous Gene; Human; Image; Industrialization; Influentials; Lateral; Learning; Link; Literature; Mentorship; Methodology; Methods; Modeling; Modernization; Natural experiment; Neurologic Deficit; Neurons; Neurosciences; Patients; Population; Prevalence; Psychophysics; Recurrence; Research; Safety; Shapes; Stimulus; Stream; Sum; Syndrome; Techniques; Technology; Testing; Toy; Training; Vision; Visual; Visual Agnosias; Visual System; Work; artificial neural network; career; cognitive neuroscience; computational neuroscience; deep learning; deep neural network; experience; experimental study; flexibility; human model; improved; neural; neuroimaging; neurophysiology; novel; object recognition; open source; operation; presynaptic; response; synergism; theories; vision science; visual search

Project Summary How does the human brain encode visual feature conjunctions? Two influential, yet disparate, research traditions have proposed two different mechanisms. Research in neurophysiology and computer vision has explored static conjunction coding, where feature conjunctions are automatically extracted via a feedforward hierarchy. While efficient, this mechanism may be limited to encoding conjunctions enabled by the hierarchy's learned connectivity. By contrast, research in cognitive psychology has explored dynamic conjunction coding, which sequentially encodes task-relevant conjunctions via attentional selection. While slow, this mechanism is flexible and capable of encoding any feature conjunction. Despite evidence for both mechanisms, their interplay remains unclear. In this project, we leverage advances in deep learning and open neuroimaging datasets to understand how these two mechanisms interact in the human brain, yielding the best of both worlds: fast, but flexible conjunction coding. Through the three complementary Specific Aims, we advance our understanding of these fundamental issues in vision science, and on a practical level, develop approaches that can be used to improve computer vision, aiding the development of useful technologies like autonomous vehicles. In the course of this project, I will master modern approaches in deep learning and computational neuroscience through the mentorship of my sponsor Dr. Kriegeskorte and my co-sponsor Dr. Fusi, equipping me for a career leading a lab that bridges cognitive science, neuroscience, and artificial intelligence. Hypotheses: The human brain implements static conjunction coding via neural populations with “and-like” tuning to feature combinations that emerges via feedforward convergence of neurons tuned to single features, while dy- namic conjunction coding requires recurrent connections. Static conjunction coding can rapidly encode familiar, but not unfamiliar conjunctions, while dynamic conjunction coding can encode any conjunction, but more slowly. Aim 1: We use a massive open-source fMRI dataset to chart the prevalence of static conjunction coding through- out the human visual system, using a method we developed in preliminary analyses. Aim 2: We apply “synthetic neurophysiology” to feedforward artificial neural networks to understand how static conjunction coding emerges in a feedforward hierarchy, beginning by characterizing conjunction-tuned units iden- tified in preliminary analyses, followed by testing influential models for how this occurs in biological vision. Aim 3: We test whether feedforward artificial neural networks exhibit similar limitations on a visual search task compared to known results for human feedforward vision, followed by testing whether introducing recurrent con- nections to networks can overcome these limitations.

项目摘要 人类大脑如何编码视觉特征连词？两个具有代表性但截然不同的研究传统 提出了两种不同的机制。神经生理学和计算机视觉的研究已经探索了静态 合取编码，其中特征合取通过前馈层次自动提取。而 有效地，该机制可以限于编码由层次结构的学习的连接性启用的合取。 相比之下，认知心理学的研究探索了动态连接编码， 通过注意力选择来编码任务相关的连接词。虽然缓慢，但这种机制是灵活的， 编码任何特征连接。尽管有证据表明这两种机制，但它们之间的相互作用仍不清楚。在 在这个项目中，我们利用深度学习和开放神经成像数据集的进步来了解这些 人脑中有两种机制相互作用，产生两全其美的结果：快速但灵活的合取编码。 通过三个互补的具体目标，我们推进我们对这些基本问题的理解， 视觉科学，并在实践层面上，开发可用于改善计算机视觉的方法， 自动驾驶汽车等有用技术的发展。在这个项目的过程中，我将掌握 通过我的赞助商Dr. Kriegeskorte和我的共同赞助人Fusi博士，为我的职业生涯提供了装备，领导了一个连接认知科学的实验室， 神经科学和阿尔蒂官方情报。 假设：人脑通过具有“与”调谐的神经群实现静态连接编码 通过调整到单一特征的神经元的前馈收敛而出现的特征组合，而dy- 动态连接编码需要循环连接。静态合取编码可以快速编码熟悉的， 但不包括不熟悉的连词，而动态连词编码可以编码任何连词，但速度更慢。 目的1：我们使用大量的开源fMRI数据集来绘制静态连接编码的流行率，通过- 人类视觉系统，使用我们在初步分析中开发的方法。 目标2：我们将“合成神经生理学”应用于前馈阿尔蒂正式神经网络，以了解静态 合取编码出现在前馈层次结构中，首先描述合取调谐单元iden， 在初步分析中确定了艾德，然后在基本模型中测试这在生物视觉中是如何发生的。 目标3：我们测试前馈阿尔蒂正式神经网络是否在视觉搜索任务中表现出类似的限制 与人类前馈视觉的已知结果相比，随后测试是否引入经常性的控制， 与网络的连接可以克服这些限制。