Internet Move Brain (IMBD): Using movies and machine learning competitions to understand how the brain supports natural behaviour

我们对大脑在自然条件下如何运作，以及支持扩展的现实世界行为（如语言理解或情感处理）的各种雨网络如何组织和相互作用，几乎没有科学的理解。克服这个问题将为人工智能以及我们如何诊断精神疾病的未来急需的发展提供信息，同时也有助于生物学其他领域中处理高度复杂系统（如大脑）的数学方法的发展。目前的项目提出了解决方案的一部分，即拥有足够数量的数据（正确类型的数据），并开发新的数学方法，以更好地考虑大脑功能的复杂性。我们可以通过生成第一个功能性磁共振成像（fMRI）数据库来实现这一目标，这些数据库来自于通过完整长度的电影引发的参与自然行为功能的人。该项目的范围扩展到自动注释电影，使eh行为，功能磁共振成像和注释数据公开，通过机器学习竞赛进行众包分析，并利用数据探索实时脑机交互和新数字界面的设计。通过神经成像实验，我们在理解大脑功能方面取得了很大进展，将一般行为分解为与特定大脑区域活动相关的离散过程。然而，认知神经科学需要一种新的方法来产生这一领域的基础性进展，而不是增量知识。该项目的核心是通过开发适当的数学模型来研究生物系统中与事件相关的数据，从而在宏观层面上研究大脑功能。人类连接组项目提供了1200名参与者在“休息”一小时后扫描的数据，帮助我们通过功能磁共振成像推进了对功能连接的理解。目前的项目将为认知神经科学界产生类似的有价值的数据，允许人们在参与电影引发的自然过程的同时研究大脑网络的组织。参与者将观看100部他们以前从未看过的未剪辑英语电影之一。其中包括十部精心挑选的电影，分别来自十种类型（动作，喜剧，戏剧，幻想恐怖，音乐，神秘，浪漫，科幻，战争）。在考虑销售和综合评论的成功衡量标准上，电影将获得很高的分数。完整的数据库最好由600名参与者组成。在为期4年的博士项目结束时，可以实现一个中等规模的数据库，并每年举办机器学习竞赛，以分配寻找解码与动作注释相关的大脑网络的最佳方法的过程。行为、功能磁共振成像和注释数据将通过一个定制的网络应用程序公开提供。该网站还将提供浏览功能磁共振成像解码结果并进行进一步分析的工具。这些创新将迅速加速了解人类大脑在自然条件下如何运作的必要进展，并具有医疗，教育和商业应用。除了数据收集，该项目还涉及两个令人兴奋的领域：全长电影的自动注释和现实生活中大脑功能的图论模型。多学科项目的完成将提供一种新的方法来处理认知神经科学问题，并提供适量的信息数据，以开始理解真实的生活条件下的大脑。在整个过程中，数学和生物技术将具有同等的用途，并将提供一个概念证明，我们可以从根本上改变我们探索大脑功能的方式，通过考虑过去30年来在神经成像方面学到的经验教训，并将这些见解转化为实验室以外的实际应用。

We have little to no scientific understanding of how the brain operates in natural conditions or how various rain networks that support extended real-world behaviours, like language comprehension or emotional processing, organise and interact. Overcoming this issue would inform future much-need developments in Artificial Intelligence and in how we diagnose mental illness, while also contributing to the developments of mathematical approaches in other areas of biology that deal with highly complex systems like brains. The current project proposed that part of the solution of having a sufficient amount of data (of the right type) and developing new mathematical approaches that better consider the complexity of brain function. We can achieve this by generating the first functional magnetic resonance imaging (fMRI) database from people engaged in natural behavioural functions as elicited through full-length movies. the Scope of the project extents to automatically annotating movies, making eh behavioural, fMRI and annotation data publicly available, crowdsourcing analyses via machine learning competitions and making use of the data to explore real-time brain-computer interaction and the design of new digital interfaces. Much progress towards understanding brain function has been made through neuroimaging experiments by decomposing general behaviours into discrete processes that can be associated with activity in particular brain regions. However, a new approach in cognitive neuroscience is needed to produce foundational advances in this field, instead of incremental knowledge. The core of this project is to focus on studying brain function at the macroscopic level by developing appropriate mathematical models to study event-related data from a biological system. The Human Connectome Project, which provides data from 1200 participants scanned for an hour 'at rest' has helped us advance our understanding of functional connectivity via fMRI. The current project would produce similarly valuable data for the cognitive neuroscience community, allowing one to study the organisation of brain networks while engaging in natural processes as elicited by movies. Participants will watch one of 100 uncut English language movies that they have not previously seen. These include ten carefully chosen movies from each of the ten genres (action, comedy, drama, fantasy horror, musical, mystery, romance, sci-fi, war). Movies will have scored highly on a metric of success that considers sales and aggregated reviews. The full database would ideally consist of 600 participants. By the end of a 4-year doctoral project, a medium-sized database is achievable, as well as running yearly machine learning competitions to distribute the process of finding the best methods to decode brain networks associated with annotations of the moves. Behavioural, fMRI and annotation data will be made publicly available through a custom web application. This website will also have the tools to browse and conduct further analyses with decoded fMRI results.These innovations will rapidly accelerate needed advances in understanding how the human brain operates under natural conditions and has medical, education and commercial applications. Aside from data collection, the project involves advancing two exciting areas: automatic annotation of full-length movies and graph theory models of real-life brain function. The completion of the multi-disciplinary project will provide a new way of approaching cognitive neuroscience questions and an appropriate amount of informative data to start understanding the brain in real life conditions. Throughout, mathematical and biological techniques will be of equal use and will provide a proof of concept that we can fundamentally change how we explore brain functioning by considering lessons learnt in neuroimaging in the past 30 years and translating these insights into practical applications to be used outside of the lab.