Multimodal and Multivariate Machine Learning Methods for Nonlinearly Coupled Oscillatory Systems

学习适当的表示，或从数据中提取有用的特征，是机器学习的基本问题之一。近年来，多模态神经影像学已成为基础研究和临床诊断的重要工具。通过利用机器学习的方法，我们可以进一步理解多模态神经数据，并从大量高维数据中提取新的见解，例如从脑电图/肌电图记录和血流动力学的同时测量中获得（例如功能磁共振成像的近红外光谱）。然而，如果模态特定动力学之间的耦合是非线性的，目前使用的分析方法不能最优地提取潜在的共同因素（潜在源）。这是由于这些方法不是真正的多模态，或者它们没有充分考虑到已建立的生成模型和模态之间耦合类型的非线性性质。此外，今天的方法在准确性（例如预测或回归质量方面的错误）和可解释性（即根据隐藏原因/来源解释结果表示的能力）之间进行权衡。拟建项目分为两部分。在第一部分（分析）中，我们将开发新的多元方法，用于从多模态成像数据中同时提取非线性相互作用源。特别是，我们将专注于特定领域的生成模型，以便找到多模态数据的低维表示，最大限度地解释底层系统的耦合动态。同时，提取的源将遵循特定于领域的生成模型假设，因此在其中是可解释的。具体来说，我们将开发新的多模态和多元空间滤波方法，以揭示多模态神经成像数据的共同来源，这些数据的动态是非线性和非瞬时耦合的。通过寻找共同的源空间表示，我们预计我们的方法不仅将在建立测量模式之间的联系方面提供出色的性能，而且还将克服上述准确性和可解释性之间的权衡。在这个项目的第二部分，我们将把新开发的方法应用于（计算）神经科学和神经技术领域现有的开放问题。我们希望能够对以下问题做出重大贡献：(i)事件相关电位（ERP）产生的机制，（ii）脑机接口（BCI）的新型无监督训练方法，以及（iii）更好地理解脑电图频谱功率和近红外光谱测量的共同动态，这反过来将导致BCI应用的卓越性能。

Learning appropriate representations, or extracting useful features from data, is one of the fundamentalproblems of Machine Learning. Recently, multimodal neuroimaging has become an important tool forbasic research and clinical diagnosis. By utilizing methods from machine learning, it has been possibleto further our understanding of multimodal neural data and extract novel insights from the multitudeof high dimensional data, such as obtained from EEG/EMG recordings and simultaneous measuresof hemodynamics (e.g. NIRS of fMRI). However, analysis methods that are currently being used arenot able to optimally extract the underlying common factors (latent sources) if the coupling betweenthe modality specific dynamics is nonlinear. This is due to the fact that either the methods are nottruly multimodal or they do not fully take into account established generative models and the nonlinearnature of the types of coupling between modalities. Furthermore, todays methods suffer from a tradeoffbetween accuracy (e.g. errors in terms of prediction or quality of regression) and interpretability (i.e. theability to interpret the resulting representations with respect to the hidden causes/sources).The proposed project is organized in two parts. In the first (analytical) part we will develop novelmultivariate methods for the simultaneous extraction of nonlinearly interacting sources from multimodalimaging data. In particular we will focus on domain specific generative models in order to find low dimensional representations of the multimodal data that maximally explain the coupled dynamics of theunderlying system. At the same time, the extracted sources will adhere to the domain specific generativemodel assumptions and will therefore be interpretable therein. Specifically we will develop novelmultimodal and multivariate spatial filtering methods that uncover common sources in multimodal neuroimaging data whose dynamics are nonlinearly and nonstantaneously coupled. By seeking common source space representations, we anticipate that our methods will not only provide excellent performance in terms of establishing a connection between measurement modalities, but that they will overcome the aforementioned tradeoff between accuracy and interpretability.In the second part of this project we will apply the newly developed methods to existing open questionsfrom the fields of (computational) neuroscience and neurotechnology. We expect to be able tocontribute substantially to questions concerning (i) the mechanisms underlying the generation of eventrelated potentials (ERP), (ii) novel unsupervised training methods for Brain-Computer Interfaces (BCI), and (iii) better understanding of common dynamics in EEG spectral power and NIRS measurements which in turn will lead to superior performance of BCI applications.