CRCNS US-France Research Proposal: Oscillatory processes for visual reasoning in deep neural networks

CRCNS 美国-法国研究提案：深度神经网络中视觉推理的振荡过程

• 批准号: 1912280
• 负责人: Thomas Serre
• 金额: $ 54.88万
• 依托单位: Brown University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-12-01 至 2023-11-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1912280&HistoricalAwards=false
• 关键词: CRCNS; US; France; Research; Proposal

The development of deep convolutional networks (DCNs) has recently led to great successes in machine vision. Despite these successes, to date, the most impressive results have been obtained for image categorization tasks such as indicating whether an image contains a particular object. However, DCNs ability to solve more complex visual reasoning problems such as understanding the visual relations between objects remains limited. Interestingly, much work in computer vision is currently being devoted to extending DCNs, but these models are still outmatched by the power and versatility of the brain, perhaps in part due to the richer neuronal computations available to cortical circuits. The challenge is to identify which neuronal mechanisms are relevant and to find suitable abstractions to model them. One promising set of candidates is the neural oscillations that are found throughout the brain. This project seeks to identify the key oscillatory components and characterize the neural computations underlying humans ability to solve visual reasoning tasks, and to use similar strategies in modern deep learning architectures.This project will use existing computational models to develop tasks and stimuli to be used in EEG studies to identify the key oscillatory components underlying human visual reasoning ability. The analysis of these EEG data will be guided by the development of a biophysically-realistic computational neuroscience model. This will inform the development of hypotheses on the circuit mechanisms underlying the oscillatory clusters and relate these mechanisms to neural computations. Finally, the project will develop novel machine learning idealizations of these neural computations, which are trainable with current deep learning methods but still interpretable at the neural circuit level. In particular, the project will further develop initial machine learning formulation of oscillations based on complex-valued neuronal units, thus extending the approach and demonstrating its ability to qualitatively capture key oscillatory processes underlying visual reasoning. A companion project is being funded by the French National Research Agency (ANR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度卷积网络（DCN）的发展最近在机器视觉领域取得了巨大成功。尽管取得了这些成功，迄今为止，最令人印象深刻的结果已经获得了图像分类任务，如指示图像是否包含特定对象。然而，DCN解决更复杂的视觉推理问题（如理解对象之间的视觉关系）的能力仍然有限。有趣的是，计算机视觉领域的许多工作目前都致力于扩展DCN，但这些模型仍然无法与大脑的能力和多功能性相匹配，这可能部分是由于皮层电路可以进行更丰富的神经元计算。挑战在于确定哪些神经元机制是相关的，并找到合适的抽象来建模。一组有希望的候选者是在整个大脑中发现的神经振荡。 该项目旨在识别关键的振荡成分，并描述人类解决视觉推理任务能力的神经计算特征，并在现代深度学习架构中使用类似的策略。该项目将使用现有的计算模型开发用于EEG研究的任务和刺激，以识别人类视觉推理能力的关键振荡成分。这些EEG数据的分析将受到生物物理现实计算神经科学模型的开发的指导。这将告知发展的假设振荡集群的电路机制，并将这些机制与神经计算。最后，该项目将开发这些神经计算的新型机器学习理想化，这些机器学习理想化可以使用当前的深度学习方法进行训练，但仍然可以在神经电路级别进行解释。 特别是，该项目将进一步开发基于复值神经元单元的振荡的初始机器学习公式，从而扩展该方法并展示其定性捕获视觉推理基础上的关键振荡过程的能力。法国国家研究机构（ANR）正在资助一个配套项目。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Same-different conceptualization: a machine vision perspective

• DOI: 10.1016/j.cobeha.2020.08.008
• 发表时间: 2021-02-01
• 期刊: CURRENT OPINION IN BEHAVIORAL SCIENCES
• 影响因子: 5
• 作者: Ricci, Matthew;Cadene, Remi;Serre, Thomas
• 通讯作者: Serre, Thomas

Disentangling neural mechanisms for perceptual grouping

• DOI: 10.32470/ccn.2019.1130-0
• 发表时间: 2019-06
• 期刊: ArXiv
• 作者: Junkyung Kim;Drew Linsley;Kalpit C. Thakkar;Thomas Serre

Diffusion Models as Artists: Are we Closing the Gap between Humans and Machines?

• DOI: 10.48550/arxiv.2301.11722
• 发表时间: 2023-01
• 作者: Victor Boutin;Thomas Fel;Lakshya Singhal;Rishav Mukherji;Akash Nagaraj;Julien Colin;Thomas Serre

Learning Functional Transduction

• DOI: 10.48550/arxiv.2302.00328
• 发表时间: 2023-02
• 期刊: ArXiv
• 作者: Mathieu Chalvidal;Thomas Serre;Rufin VanRullen

Go with the flow: Adaptive control for Neural ODEs

• 发表时间: 2020-06
• 期刊: arXiv: Learning
• 作者: Mathieu Chalvidal;Matthew Ricci;R. VanRullen;Thomas Serre