Deep learning for neural decoding in motor cortex
Deep learning for neural decoding in motor cortex
复制标题
深度学习用于运动皮层中的神经解码
DOI：
10.1088/1741-2552/ac8fb5
复制
发表时间：
2022-10-01
影响因子：
4
通讯作者：
Wang, Linbing
中科院分区：
文献类型：
--
作者：
Liu, Fangyu;Meamardoost, Saber;Wang, Linbing
Objective.神经解码是神经工程和神经数据分析的重要工具。在用于神经解码的各种机器学习算法中，最近引入的深度学习有望脱颖而出。因此，我们试图应用深度学习从运动皮层神经元的活动中解码运动轨迹。Approach.在本文中，我们评估了深度学习方法在三种不同解码方案（并发、时延和时空）中的性能。在并发解码方案中，网络的输入是与运动一致的神经活动，包括人工神经网络（ANN）和长短期记忆（LSTM）在内的深度学习网络被应用于解码运动，并与传统的机器学习算法进行了比较。ANN和LSTM都在时延解码方案中进行了进一步评估，其中神经信号和运动之间允许时间延迟。最后，在时空解码方案中，我们训练卷积神经网络（CNN）从代表神经元空间排列的图像中提取运动信息，它们的活动和连接体（即神经元之间连接的相对强度），并将CNN和ANN结合起来开发混合时空网络。为了揭示深度学习发现的用于运动解码的混合网络中CNN的输入特征，我们进行了灵敏度分析并识别了空间域中的特定区域。主要结果。深度学习网络（ANN和LSTM）在并发解码方案中优于传统机器学习算法。ANN和LSTM在时延解码方案中的结果表明，当神经活动和运动之间的时间关系随时间动态变化时，包括运动之前时间点的神经数据使解码器能够更稳健地执行。在时空解码方案中，包含并发ANN解码器的混合时空网络的性能优于单网络并发解码器。意义总之，我们的研究表明，深度学习可以成为一种强大而有效的行为神经解码方法。
Objective. Neural decoding is an important tool in neural engineering and neural data analysis. Of various machine learning algorithms adopted for neural decoding, the recently introduced deep learning is promising to excel. Therefore, we sought to apply deep learning to decode movement trajectories from the activity of motor cortical neurons. Approach. In this paper, we assessed the performance of deep learning methods in three different decoding schemes, concurrent, time-delay, and spatiotemporal. In the concurrent decoding scheme where the input to the network is the neural activity coincidental to the movement, deep learning networks including artificial neural network (ANN) and long-short term memory (LSTM) were applied to decode movement and compared with traditional machine learning algorithms. Both ANN and LSTM were further evaluated in the time-delay decoding scheme in which temporal delays are allowed between neural signals and movements. Lastly, in the spatiotemporal decoding scheme, we trained convolutional neural network (CNN) to extract movement information from images representing the spatial arrangement of neurons, their activity, and connectomes (i.e. the relative strengths of connectivity between neurons) and combined CNN and ANN to develop a hybrid spatiotemporal network. To reveal the input features of the CNN in the hybrid network that deep learning discovered for movement decoding, we performed a sensitivity analysis and identified specific regions in the spatial domain. Main results. Deep learning networks (ANN and LSTM) outperformed traditional machine learning algorithms in the concurrent decoding scheme. The results of ANN and LSTM in the time-delay decoding scheme showed that including neural data from time points preceding movement enabled decoders to perform more robustly when the temporal relationship between the neural activity and movement dynamically changes over time. In the spatiotemporal decoding scheme, the hybrid spatiotemporal network containing the concurrent ANN decoder outperformed single-network concurrent decoders. Significance. Taken together, our study demonstrates that deep learning could become a robust and effective method for the neural decoding of behavior.