Multimodal fusion of EEG-fNIRS: a mutual information-based hybrid classification framework
Multimodal fusion of EEG-fNIRS: a mutual information-based hybrid classification framework
复制标题
基于互信息的EEG-fNIRS多模态融合混合分类框架
DOI：
10.1364/boe.413666
复制
发表时间：
2021-03-01
影响因子：
3.4
通讯作者：
Shahriari, Yalda
中科院分区：
文献类型：
--
作者：
Deligani, Roohollah Jafari;Borgheai, Seyyed Bahram;Shahriari, Yalda
多模态数据融合是当前主要的神经影像研究方向之一，通过利用不同模态的互补信息来克服个体模态的根本局限性。脑电图（EEG）和功能性近红外光谱（fNIRS）是特别引人注目的方式，由于其潜在的互补功能，反映了神经反应的电血流动力学特征。然而，目前的多模态研究缺乏一个全面的系统的方法来适当地合并的互补功能，从他们的多模态数据。确定一种系统的方法来正确融合EEG-fNIRS数据并利用其互补潜力对于提高性能至关重要。本文提出了一个框架，融合EEG-fNIRS数据在特征级分类，依靠互信息为基础的特征选择方法与功能之间的互补性。目标是优化多模态特征之间的互补性、冗余性和相关性，这些特征相对于属于病理状况或健康对照的类别标签。9名肌萎缩侧索硬化症（ALS）患者和9名对照者在视觉-心理任务期间进行了多模态数据记录。提取多个光谱和时间特征，并将其输入特征选择算法，然后输入分类器，该分类器通过交叉验证过程选择优化的特征子集。结果表明，大大提高了混合分类性能相比，个别模式和传统的分类相比，没有特征选择，这表明我们提出的框架更广泛的神经临床应用的潜在功效。(c)根据OSA开放获取出版协议的条款，2021年美国光学学会
Multimodal data fusion is one of the current primary neuroimaging research directions to overcome the fundamental limitations of individual modalities by exploiting complementary information from different modalities. Electroencephalography (EEG) and functional near infrared spectroscopy (fNIRS) are especially compelling modalities due to their potentially complementary features reflecting the electro-hemodynamic characteristics of neural responses. However, the current multimodal studies lack a comprehensive systematic approach to properly merge the complementary features from their multimodal data. Identifying a systematic approach to properly fuse EEG-fNIRS data and exploit their complementary potential is crucial in improving performance. This paper proposes a framework for classifying fused EEG-fNIRS data at the feature level, relying on a mutual information-based feature selection approach with respect to the complementarity between features. The goal is to optimize the complementarity, redundancy and relevance between multimodal features with respect to the class labels as belonging to a pathological condition or healthy control. Nine amyotrophic lateral sclerosis (ALS) patients and nine controls underwent multimodal data recording during a visuo-mental task. Multiple spectral and temporal features were extracted and fed to a feature selection algorithm followed by a classifier, which selected the optimized subset of features through a cross-validation process. The results demonstrated considerably improved hybrid classification performance compared to the individual modalities and compared to conventional classification without feature selection, suggesting a potential efficacy of our proposed framework for wider neuro-clinical applications.(c) 2021 Optical Society of America under the terms of the OSA Open Access Publishing Agreement