Classification of fNIRS Finger Tapping Data With Multi-Labeling and Deep Learning
Classification of fNIRS Finger Tapping Data With Multi-Labeling and Deep Learning
复制标题
利用多重标记和深度学习对 fNIRS 手指敲击数据进行分类
DOI：
10.1109/jsen.2021.3115405
复制
发表时间：
2021
影响因子：
4.3
通讯作者：
Hirshfield, Leanne
中科院分区：
文献类型：
--
作者：
Sommer, Natalie M.;Kakillioglu, Burak;Grant, Trevor;Velipasalar, Senem;Hirshfield, Leanne
研究大脑和手指敲击运动之间的关系有助于更好地理解神经肌肉损伤。此外，在手指敲击练习期间非侵入性地获取大脑数据信号，并建立鲁棒的分类模型可以帮助人机交互领域。在本文中，我们提出了一个很有前途的方法，空间描述性的多标记的时空功能近红外光谱（fNIRS）数据，自主检测不同的手指敲击水平在大脑的不同区域同时。我们的多类多标签技术为左右食指分配标签，给定的标签描述了三种不同的手指敲击频率（休息，80 bpm和120 bpm）之一，在大脑运动皮层的相应对侧空间位置进行监测。我们训练了一个基于CNN/LSTM的网络，以在空间上同时对上述手指敲击水平进行分类。基于两个大脑区域的同时多标记预测的评估是用多标记中常用的度量、汉明损失以及基于混淆矩阵的测量沿着来执行的。取得了令人满意的测试结果，平均汉明损失为0.185，平均F分数为0.81，平均准确度为0.81。此外，我们通过生成Shapley加法解释值并将其绘制在类似图像的背景上来解释我们的模型和新颖的多标记方法，该背景代表用作数据输入的fNIRS通道布局。Shapley值有助于为我们的深度学习模型增加可解释性，并通过确认预期结果，为未来开发复杂的深度学习模型提供了一条途径，这些模型试图预测社会认知情感状态。
Studying the relationship between the brain and finger tapping motions can contribute towards an improved understanding of neuromuscular impairment. Furthermore, acquiring brain data signals non-intrusively during finger tapping exercises, and building a robust classification model can aid in the field of human computer interaction. In this paper, we present a promising approach for spatially descriptive multi-labeling of spatiotemporal functional Near Infrared Spectroscopy (fNIRS) data to autonomously detect different finger tapping levels in different regions of the brain simultaneously. Our multi-class multi-labeling technique assigns labels to the left and right index fingers, and a given label describes one of three different finger tapping frequencies (rest, 80bpm, and 120bpm) to be monitored in the corresponding contralateral spatial location in the brain’s motor cortex. We train a CNN/LSTM-based network to classify the aforementioned finger tapping levels spatially and simultaneously. The evaluation, based on simultaneous multi-label predictions for two brain regions, is performed with a metric commonly used in multi-labeling, Hamming Loss, along with confusion matrix-based measurements. Promising testing results are obtained with an average Hamming Loss of 0.185, average F-Score of 0.81, and average Accuracy of 0.81. Moreover, we explain our model and novel multi-labeling approach by generating Shapley Additive Explanation values and plotting them on an image-like background, which represents the fNIRS channel layout used as data input. Shapley values help to add interpretability to our deep learning model and by confirming expected results, offer a pathway to the future development of complex deep learning models that attempt to predict social-cognitive-affective states.