Combining Modalities with Different Latencies for Optimal Motor Control
Combining Modalities with Different Latencies for Optimal Motor Control
复制标题
DOI：
10.1162/jocn.2008.20133
复制
发表时间：
2008-11-01
影响因子：
3.2
通讯作者：
Hikosaka, Okihide
中科院分区：
文献类型：
--
作者：
Bissmarck, Fredrik;Nakahara, Hiroyuki;Hikosaka, Okihide
反馈信号可以具有不同的模态、延迟和准确性。为了学习和控制运动任务，可用的反馈可能是多余的，并且没有必要依赖于每个可访问的反馈回路。那么应该使用哪些反馈回路？在这篇文章中，我们提出，延迟是一个关键因素，以确定哪些信号将在不同的学习阶段的影响。我们使用一个计算框架来研究反馈模块的作用，在最佳电机控制不同的lavelets。强化学习算法学习依赖于更有用的模块，而不是模块之间的显式门控。我们测试了两种不同实现的范例，这证实了我们的假设。首先，我们研究了反馈延迟如何影响两个相同模块的竞争力。在第二部分中，我们研究了视觉序列学习的一个例子，其中一个可塑的、更快的躯体感觉模块与一个预先获得的、更慢的视觉模块相互作用。我们发现，整体性能仅取决于较快模块的延迟，而相对延迟决定了较快模块与较慢模块的独立性。在第二种实现中，延迟较短的体感模块超越了较慢的视觉模块，并实现了更好的整体性能。视觉模块在早期和晚期学习中发挥着不同的作用。首先，它是探索躯体感觉模块的指南。然后，当学习收敛时，它有助于对系统噪声和外部扰动的鲁棒性。总的来说，这些结果表明，我们的框架成功地学会了利用最有用的反馈进行最优控制。
Feedback signals may be of different modality, latency, and accuracy. To learn and control motor tasks, the feedback available may be redundant, and it would not be necessary to rely on every accessible feedback loop. Which feedback loops should then be utilized? In this article, we propose that the latency is a critical factor to determine which signals will be influential at different learning stages. We use a computational framework to study the role of feedback modules with different latencies in optimal motor control. Instead of explicit gating between modules, the reinforcement learning algorithm learns to rely on the more useful module. We tested our paradigm for two different implementations, which confirmed our hypothesis. In the first, we examined how feedback latency affects the competitiveness of two identical modules. In the second, we examined an example of visuomotor sequence learning, where a plastic, faster somatosensory module interacts with a preacquired, slower visual module. We found that the overall performance depended on the latency of the faster module alone, whereas the relative latency determines the independence of the faster from the slower. In the second implementation, the somatosensory module with shorter latency overtook the slower visual module, and realized better overall performance. The visual module played different roles in early and late learning. First, it worked as a guide for the exploration of the somatosensory module. Then, when learning had converged, it contributed to robustness against system noise and external perturbations. Overall, these results demonstrate that our framework successfully learns to utilize the most useful available feedback for optimal control.