Internet Move Brain (IMBD): Using movies and machine learning competitions to understand how the brain supports natural behaviour

对于大脑在自然条件下是如何运作的，以及支持扩展的现实世界行为（如语言理解或情感处理）的各种神经网络是如何组织和相互作用的，我们几乎没有科学的理解。克服这个问题将为未来人工智能和精神疾病诊断领域急需的发展提供信息，同时也有助于在处理大脑等高度复杂系统的其他生物学领域发展数学方法。目前的项目提出了部分解决方案，即拥有足够数量的（正确类型的）数据，并开发新的数学方法，更好地考虑大脑功能的复杂性。我们可以通过生成第一个功能性磁共振成像（fMRI）数据库来实现这一目标，这些数据库来自于通过全长电影激发的自然行为功能。该项目的范围扩展到自动注释电影，公开eh行为，功能磁共振成像和注释数据，通过机器学习竞赛进行众包分析，利用数据探索实时脑机交互和新数字接口的设计。通过神经成像实验，将一般行为分解为与特定大脑区域的活动相关的离散过程，在理解大脑功能方面取得了很大进展。然而，认知神经科学需要一种新的方法来产生这个领域的基础进展，而不是增量知识。该项目的核心是通过建立适当的数学模型来研究生物系统中与事件相关的数据，在宏观层面上研究大脑功能。人类连接组项目（Human Connectome Project）提供了1200名参与者“休息”一小时的扫描数据，帮助我们通过fMRI加深了对功能连接的理解。目前的项目将为认知神经科学界提供同样有价值的数据，允许人们在参与由电影引发的自然过程时研究大脑网络的组织。参与者将观看100部他们以前没有看过的未剪辑的英语电影中的一部。其中包括从10种类型（动作、喜剧、戏剧、奇幻恐怖、音乐、神秘、爱情、科幻、战争）中精心挑选的10部电影。电影将在考虑销售和综合评价的成功指标上获得很高的分数。理想情况下，完整的数据库包含600个参与者。在一个为期4年的博士项目结束时，一个中等规模的数据库是可以实现的，同时每年举办一次机器学习竞赛，以分发寻找解码与移动注释相关的大脑网络的最佳方法的过程。行为、功能磁共振成像和注释数据将通过一个定制的网络应用程序公开提供。该网站还将提供浏览工具，并对解码后的fMRI结果进行进一步分析。这些创新将迅速加速了解人类大脑在自然条件下如何运作的必要进展，并具有医学、教育和商业应用。除了数据收集，该项目还涉及推进两个令人兴奋的领域：全长电影的自动注释和现实生活中大脑功能的图论模型。该多学科项目的完成将为解决认知神经科学问题提供新的途径，并为开始了解现实生活条件下的大脑提供适当数量的信息数据。在整个过程中，数学和生物技术将同等使用，并将提供一个概念证明，我们可以从根本上改变我们探索大脑功能的方式，通过考虑过去30年在神经成像方面的经验教训，并将这些见解转化为实验室外使用的实际应用。

We have little to no scientific understanding of how the brain operates in natural conditions or how various rain networks that support extended real-world behaviours, like language comprehension or emotional processing, organise and interact. Overcoming this issue would inform future much-need developments in Artificial Intelligence and in how we diagnose mental illness, while also contributing to the developments of mathematical approaches in other areas of biology that deal with highly complex systems like brains. The current project proposed that part of the solution of having a sufficient amount of data (of the right type) and developing new mathematical approaches that better consider the complexity of brain function. We can achieve this by generating the first functional magnetic resonance imaging (fMRI) database from people engaged in natural behavioural functions as elicited through full-length movies. the Scope of the project extents to automatically annotating movies, making eh behavioural, fMRI and annotation data publicly available, crowdsourcing analyses via machine learning competitions and making use of the data to explore real-time brain-computer interaction and the design of new digital interfaces. Much progress towards understanding brain function has been made through neuroimaging experiments by decomposing general behaviours into discrete processes that can be associated with activity in particular brain regions. However, a new approach in cognitive neuroscience is needed to produce foundational advances in this field, instead of incremental knowledge. The core of this project is to focus on studying brain function at the macroscopic level by developing appropriate mathematical models to study event-related data from a biological system. The Human Connectome Project, which provides data from 1200 participants scanned for an hour 'at rest' has helped us advance our understanding of functional connectivity via fMRI. The current project would produce similarly valuable data for the cognitive neuroscience community, allowing one to study the organisation of brain networks while engaging in natural processes as elicited by movies. Participants will watch one of 100 uncut English language movies that they have not previously seen. These include ten carefully chosen movies from each of the ten genres (action, comedy, drama, fantasy horror, musical, mystery, romance, sci-fi, war). Movies will have scored highly on a metric of success that considers sales and aggregated reviews. The full database would ideally consist of 600 participants. By the end of a 4-year doctoral project, a medium-sized database is achievable, as well as running yearly machine learning competitions to distribute the process of finding the best methods to decode brain networks associated with annotations of the moves. Behavioural, fMRI and annotation data will be made publicly available through a custom web application. This website will also have the tools to browse and conduct further analyses with decoded fMRI results.These innovations will rapidly accelerate needed advances in understanding how the human brain operates under natural conditions and has medical, education and commercial applications. Aside from data collection, the project involves advancing two exciting areas: automatic annotation of full-length movies and graph theory models of real-life brain function. The completion of the multi-disciplinary project will provide a new way of approaching cognitive neuroscience questions and an appropriate amount of informative data to start understanding the brain in real life conditions. Throughout, mathematical and biological techniques will be of equal use and will provide a proof of concept that we can fundamentally change how we explore brain functioning by considering lessons learnt in neuroimaging in the past 30 years and translating these insights into practical applications to be used outside of the lab.