Generalized neural decoders for transfer learning across participants and recording modalities
Generalized neural decoders for transfer learning across participants and recording modalities
复制标题
DOI：
10.1088/1741-2552/abda0b
复制
发表时间：
2021-04-01
影响因子：
4
通讯作者：
Brunton, Bingni W.
中科院分区：
文献类型：
--
作者：
Peterson, Steven M.;Steine-Hanson, Zoe;Brunton, Bingni W.
目标。神经解码的进步使脑机接口能够执行越来越复杂和与临床相关的任务。然而，这种解码器通常是为特定的参与者、日期和录制地点量身定做的，限制了它们的实际长期使用。因此，一个根本的挑战是开发神经解码器，它可以对汇集的多参与者数据进行强有力的训练，并推广到新参与者。接近。我们介绍了一个新的解码器，HTNet，它使用卷积神经网络，具有两个创新：(A)希尔伯特变换，以数据驱动的频率计算频谱功率；(B)一个层，将电极水平的数据投影到预定义的大脑区域。投影层关键地实现了脑电皮质成像(ECoG)的应用，在这种情况下，电极位置不是标准化的，而且参与者之间的差异很大。我们训练HTNet使用从12名参与者中收集的11个ECoG数据来解码手臂动作，并测试看不见的ECoG或脑电(EEG)参与者的表现；这些预先训练的模型随后也针对每个测试参与者进行了微调。主要结果。HTNet在对看不见的参与者进行测试时，即使使用了不同的记录方式，也比最先进的解码器性能更好。通过微调这些通用HTNet解码器，我们获得了接近最好的定制解码器的性能，只需50次ECoG或20次EEG事件。我们还能够解释HTNet训练有素的体重，并展示其提取生理相关特征的能力。意义重大。通过推广到新的参与者和记录模式，稳健地处理电极放置的变化，并允许以最少的数据针对参与者进行微调，与当前最先进的解码器相比，HTNet适用于更广泛的神经解码应用。
Objective. Advances in neural decoding have enabled brain-computer interfaces to perform increasingly complex and clinically-relevant tasks. However, such decoders are often tailored to specific participants, days, and recording sites, limiting their practical long-term usage. Therefore, a fundamental challenge is to develop neural decoders that can robustly train on pooled, multi-participant data and generalize to new participants. Approach. We introduce a new decoder, HTNet, which uses a convolutional neural network with two innovations: (a) a Hilbert transform that computes spectral power at data-driven frequencies and (b) a layer that projects electrode-level data onto predefined brain regions. The projection layer critically enables applications with intracranial electrocorticography (ECoG), where electrode locations are not standardized and vary widely across participants. We trained HTNet to decode arm movements using pooled ECoG data from 11 of 12 participants and tested performance on unseen ECoG or electroencephalography (EEG) participants; these pretrained models were also subsequently fine-tuned to each test participant. Main results. HTNet outperformed state-of-the-art decoders when tested on unseen participants, even when a different recording modality was used. By fine-tuning these generalized HTNet decoders, we achieved performance approaching the best tailored decoders with as few as 50 ECoG or 20 EEG events. We were also able to interpret HTNet's trained weights and demonstrate its ability to extract physiologically-relevant features. Significance. By generalizing to new participants and recording modalities, robustly handling variations in electrode placement, and allowing participant-specific fine-tuning with minimal data, HTNet is applicable across a broader range of neural decoding applications compared to current state-of-the-art decoders.