Real-time sensory-motor integration of hippocampal place cell replay and prefrontal sequence learning in simulated and physical rat robots for novel path optimization
Real-time sensory-motor integration of hippocampal place cell replay and prefrontal sequence learning in simulated and physical rat robots for novel path optimization
复制标题
DOI：
10.1007/s00422-020-00820-2
复制
发表时间：
2020-02-24
影响因子：
1.9
通讯作者：
Dominey, Peter Ford
中科院分区：
文献类型：
--
作者：
Cazin, Nicolas;Scleidorovich, Pablo;Dominey, Peter Ford
在导航的认知层面的一个开放的问题是如何以前的探索经验进行重组，以允许创建新的有效的导航轨迹。这种行为被揭示在“旅行售货员问题”（TSP）时，大鼠发现的最短路径连接诱饵食物威尔斯经过几次探索性的遍历。我们最近发表了一个导航序列学习的模型，其中海马位置细胞的尖波涟漪重放将动物探索的最近轨迹的“片段”传输到前额叶皮层（PFC）（Cazin et al. in PLoS Comput Biol 15：e1006624，2019）。PFC被建模为一个经常性的水库网络，能够将这些片段组装成有效的序列（由位置细胞激活编码的空间位置的轨迹）。海马重放模型生成片段的分布作为其接近奖励的函数，从而实现解决TSP任务的空间信用分配的形式。综合PFC水库重建有效的TSP序列的基础上，暴露于这种分布的片段，有利于最接近奖励的路径。虽然这证明了PFC-HIPP相互作用的理论可行性，但将这种动态系统集成到实时感觉-运动系统中仍然是一个挑战。在目前的研究中，我们测试的假设，PFC水库模型可以在一个实时的感觉运动回路。因此，本文的主要目标是在模拟和真实的机器人场景中验证模型。编码模拟和物理大鼠机器人的当前位置的位置细胞激活馈送PFC库，PFC库生成表示读出中再现序列中的下一步骤的后继位置细胞激活。这是机器人的输入，机器人前进到编码位置，然后重新生成当前位置细胞激活。这就证明了化身的关键作用。如果从PFC读出的空间代码被直接回放到PFC中，则误差可能累积，并且系统可能偏离期望的轨迹。这需要一个空间滤波器将PFC代码解码到一个位置，然后重新编码该位置的新位置单元代码。在机器人中，PFC的位置单元矢量输出用于物理地移动机器人，然后生成新的位置单元编码输入到PFC，取代否则所需的软件编码过程的一部分。我们展示了这种集成的感觉-运动系统如何学习简单的导航序列，然后，重要的是，它如何根据先前的经验合成新的有效序列，如前所述（Cazin et al. 2019）。这有助于理解海马重放在新的导航序列形成和体现的重要作用。
An open problem in the cognitive dimensions of navigation concerns how previous exploratory experience is reorganized in order to allow the creation of novel efficient navigation trajectories. This behavior is revealed in the "traveling salesrat problem" (TSP) when rats discover the shortest path linking baited food wells after a few exploratory traversals. We have recently published a model of navigation sequence learning, where sharp wave ripple replay of hippocampal place cells transmit "snippets" of the recent trajectories that the animal has explored to the prefrontal cortex (PFC) (Cazin et al. in PLoS Comput Biol 15:e1006624, 2019). PFC is modeled as a recurrent reservoir network that is able to assemble these snippets into the efficient sequence (trajectory of spatial locations coded by place cell activation). The model of hippocampal replay generates a distribution of snippets as a function of their proximity to a reward, thus implementing a form of spatial credit assignment that solves the TSP task. The integrative PFC reservoir reconstructs the efficient TSP sequence based on exposure to this distribution of snippets that favors paths that are most proximal to rewards. While this demonstrates the theoretical feasibility of the PFC-HIPP interaction, the integration of such a dynamic system into a real-time sensory-motor system remains a challenge. In the current research, we test the hypothesis that the PFC reservoir model can operate in a real-time sensory-motor loop. Thus, the main goal of the paper is to validate the model in simulated and real robot scenarios. Place cell activation encoding the current position of the simulated and physical rat robot feeds the PFC reservoir which generates the successor place cell activation that represents the next step in the reproduced sequence in the readout. This is input to the robot, which advances to the coded location and then generates de novo the current place cell activation. This allows demonstration of the crucial role of embodiment. If the spatial code readout from PFC is played back directly into PFC, error can accumulate, and the system can diverge from desired trajectories. This required a spatial filter to decode the PFC code to a location and then recode a new place cell code for that location. In the robot, the place cell vector output of PFC is used to physically displace the robot and then generate a new place cell coded input to the PFC, replacing part of the software recoding procedure that was required otherwise. We demonstrate how this integrated sensory-motor system can learn simple navigation sequences and then, importantly, how it can synthesize novel efficient sequences based on prior experience, as previously demonstrated (Cazin et al. 2019). This contributes to the understanding of hippocampal replay in novel navigation sequence formation and the important role of embodiment.