Model-Based and Model-Free Mechanisms of Human Motor Learning
Model-Based and Model-Free Mechanisms of Human Motor Learning
复制标题
DOI：
10.1007/978-1-4614-5465-6_1
复制
发表时间：
2013-01-01
期刊：
影响因子：
--
通讯作者：
Krakauer, John W.
中科院分区：
文献类型：
--
作者：
Haith, Adrian M.;Krakauer, John W.
运动学习在理论上可以被定义为在潜在的不确定或变化的环境中优化运动策略的问题。这正是强化学习领域研究的普遍问题。强化学习理论提出了两种不同的方法来解决这个一般性问题：基于模型的方法首先识别任务或环境的动态，然后使用这些知识来计算最佳移动策略。相比之下，无模型方法通过反复试验直接确定成功的政策。在这里，我们回顾现有的文献中的电机控制在这种区别。在过去的十年中，运动学习的研究一直占主导地位的研究，通过适应范式引发学习，并发现结果是一致的基于模型的框架。在这种适应范例中研究患者的行为暗示小脑是内部模型的神经基质的主要候选者，该内部模型辅助基于模型的控制。然而，越来越多的实验结果表明，并不是所有的运动学习在传统的范式可以在基于模型的框架内解释，但可以理解的一个额外的组成部分的学习驱动的无模型强化成功的行动。我们的结论是，大脑保持着不同的基于模型和无模型的学习系统，具有不同的神经基质，它们以竞争平衡的方式指导行为。
Motor learning can be framed theoretically as a problem of optimizing a movement policy in a potentially uncertain or changing environment. This is precisely the general problem studied in the field of reinforcement learning. Reinforcement learning theory proposes two distinct approaches to solving this general problem: Model-based approaches first identify the dynamics of the task or environment then use this knowledge to compute the optimal movement policy. Model-free approaches, by contrast, directly identify successful policies through a process of trial and error. Here, we review existing literature on motor control in the light of this distinction. Motor learning research in the last decade has been dominated by studies that elicit learning through adaptation paradigms and find the results to be consistent with a model-based framework. Studying the behavior of patients in such adaptation paradigms has implicated the cerebellum as prime candidate for the neural substrate of the internal models that sub serve model-based control. A growing body of experimental results, however, demonstrates that not all of motor learning in conventional paradigms can be explained within model-based frameworks, but can be understood in terms of an additional component of learning driven by model-free reinforcement of successful actions. We conclude that the brain maintains distinct model-based and model-free learning systems, with distinct neural substrates, which act in competitive balance to direct behavior.