CRCNS Research Proposal: Collaborative Research: Evaluating Machine Learning Architectures Using a Massive Benchmark Dataset of Brain Responses to Natural Scenes

机器学习技术有可能从根本上改变人类大脑的研究，但需要比传统神经科学实验中通常收集的数据更多的数据。该项目的目标是通过从人类视觉系统生成大量的大脑反应数据集，推动ML技术在神经科学研究中的应用。由此产生的数据集将免费提供给科学家，教育工作者和学生。通过一年一度的建模竞赛，神经科学家将获得先进计算方法的应用经验，ML研究人员将更深入地了解人类大脑的挑战和复杂性。建模竞赛的结果将在机器学习和神经科学研究人员和学生参加的年度会议上公布，为两个小组提供互动和讨论方法的机会。该项目将促进神经科学家和人工智能研究人员之间的开放合作，以及共享数据、想法和进步的文化。这项工作的长期目标是生成数据，这些数据将导致人类视觉系统的实验验证和计算功能强大的模型的开发。项目负责人将使用高场（7特斯拉）功能性磁共振成像（fMRI）来测量人类观察者对自然图像的广泛采样的大脑反应。具体目标如下：（1）获取、预处理和分发利用最先进成像技术的大规模高分辨率fMRI数据集。该数据集将包括大脑对大约8万张照片的反应的多个样本，这些照片来自ML社区广泛使用的图像集。(2)在认知计算神经科学会议上建立并举办一个年度竞赛，为这个丰富的数据集建模。(3)通过测试新的ML启发架构作为视觉系统的模型，弥合ML架构和人脑之间的差距。项目负责人将特别关注ML的最新发展，这些发展提出了关于背侧视觉流的新假设。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Machine learning technologies have the potential to radically transform the study of the human brain, but require far more data than is typically collected during conventional neuroscience experiments. The goal of this project is to drive the application of ML techniques to neuroscience research by generating a massive dataset of brain responses from the human visual system. The resulting dataset will be freely available to scientists, educators, and students. Through a yearly modeling competition, neuroscientists will gain experience in the application of advanced computational methods and ML researchers will gain a deeper understanding of the challenges and complexities of the human brain. Results of the modeling competition will be presented at an annual conference attended by both machine learning and neuroscience researchers and students, providing an opportunity for the two groups to interact and discuss approaches. This project will foster open collaboration between neuroscientists and artificial intelligence researchers and a culture of sharing data, ideas, and progress. The long-term goal of this work is to generate data that will lead to the development of experimentally validated and computationally powerful models of the human visual system. The project leaders will use high-field (7 Tesla) functional magnetic resonance imaging (fMRI) to measure brain responses to a broad sampling of natural images in human observers. The specific objectives are as follows: (1) Acquire, pre-process, and distribute a massive, high-resolution fMRI dataset that exploits state-of-the-art imaging techniques. The dataset will include multiple samples of brain responses to roughly eighty thousand photographs drawn from an image collection that is widely used by the ML community. (2) Establish and host an annual competition for modeling this rich dataset at the conference on Cognitive Computational Neuroscience. (3) Bridge the gap between ML architectures and the human brain by testing new ML-inspired architectures as models of the visual system. The project leaders will focus specifically on recent developments in ML that suggest new hypotheses about the dorsal visual stream.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.