Interference and shaping in sensorimotor adaptations with rewards.
Interference and shaping in sensorimotor adaptations with rewards.
复制标题
DOI：
10.1371/journal.pcbi.1003377
复制
发表时间：
2014-01
影响因子：
4.3
通讯作者：
Hansel D
中科院分区：
文献类型：
--
作者：
Darshan R;Leblois A;Hansel D
当在感觉运动转换任务中施加扰动时，受试者可以通过依赖感觉反馈或在没有这种反馈的情况下依赖奖励提供的信息来适应和保持表现。例如，在一个经典的旋转任务中，运动端点必须旋转才能到达一个固定的目标，人类受试者可以成功地适应他们的达到运动的基础上，唯一的二元奖励，虽然这证明比视觉反馈困难得多。在这里，我们调查这样一个奖励驱动的感觉运动适应过程中的最小计算模型的任务。该模型的关键假设是突触可塑性是由奖励门控的。我们研究了学习动力学如何依赖于目标的大小，运动的可变性，旋转角度和目标的数量。我们发现，当运动被扰动的多个目标，不同的目标的适应过程可以干扰破坏性或建设性的感觉刺激（目标）和重叠在他们的神经元表示之间的相似性。破坏性的干扰会导致适应的急剧减缓。作为干扰的结果，适应的时间随着目标的数量而非线性地变化。我们的分析表明，这些干扰是较弱的，如果奖励的变化与主体的表现，而不是二进制的顺利。我们展示了如何塑造奖励或塑造任务可以通过减少破坏性干扰来加速适应。我们认为，实验研究奖励驱动的感觉运动适应的动态不止一个感官刺激，可以揭示潜在的学习规则。大脑具有强大的能力来适应施加在获得的感觉运动转换上的外部扰动。在这里，我们使用了一个数学模型来研究感觉运动适应中的奖励成分。我们表明，所提供的奖励信号的形状，在实验中通常是二元的，以表明成功或失败，影响适应动态。我们证明了如何适应扰动的能力，仅仅依靠二进制奖励取决于运动的变异性，扰动的大小和阈值提供奖励。当使运动响应同时适应多个感官刺激时，由于感官刺激的神经表征的重叠以及它们之间的物理距离，响应于不同刺激的运动性能之间发生在线干扰。由于相消干扰，在物理上彼此不同的几个刺激引起扰动时，适应可能非常缓慢。当引入中间刺激时，相邻刺激之间的物理距离减小，并且可以出现建设性干扰，从而导致更快的适应。值得注意的是，在训练过程中，通过增加感觉刺激的数量，可以加速对广泛的感觉运动扰动的适应，也就是说，如果一个人学得更多，学习就会更快。
When a perturbation is applied in a sensorimotor transformation task, subjects can adapt and maintain performance by either relying on sensory feedback, or, in the absence of such feedback, on information provided by rewards. For example, in a classical rotation task where movement endpoints must be rotated to reach a fixed target, human subjects can successfully adapt their reaching movements solely on the basis of binary rewards, although this proves much more difficult than with visual feedback. Here, we investigate such a reward-driven sensorimotor adaptation process in a minimal computational model of the task. The key assumption of the model is that synaptic plasticity is gated by the reward. We study how the learning dynamics depend on the target size, the movement variability, the rotation angle and the number of targets. We show that when the movement is perturbed for multiple targets, the adaptation process for the different targets can interfere destructively or constructively depending on the similarities between the sensory stimuli (the targets) and the overlap in their neuronal representations. Destructive interferences can result in a drastic slowdown of the adaptation. As a result of interference, the time to adapt varies non-linearly with the number of targets. Our analysis shows that these interferences are weaker if the reward varies smoothly with the subject's performance instead of being binary. We demonstrate how shaping the reward or shaping the task can accelerate the adaptation dramatically by reducing the destructive interferences. We argue that experimentally investigating the dynamics of reward-driven sensorimotor adaptation for more than one sensory stimulus can shed light on the underlying learning rules. The brain has a robust ability to adapt to external perturbations imposed on acquired sensorimotor transformations. Here, we used a mathematical model to investigate the reward-based component in sensorimotor adaptations. We show that the shape of the delivered reward signal, which in experiments is usually binary to indicate success or failure, affects the adaptation dynamics. We demonstrate how the ability to adapt to perturbations by relying solely on binary rewards depends on motor variability, size of perturbation and the threshold for delivering the reward. When adapting motor responses to multiple sensory stimuli simultaneously, on-line interferences between the motor performance in response to the different stimuli occur as a result of the overlap in the neural representation of the sensory stimuli, as well as the physical distance between them. Adaptation may be extremely slow when perturbations are induced to a few stimuli that are physically different from each other because of destructive interferences. When intermediate stimuli are introduced, the physical distance between neighbor stimuli is reduced, and constructive interferences can emerge, resulting in faster adaptation. Remarkably, adaptation to a widespread sensorimotor perturbation is accelerated by increasing the number of sensory stimuli during training, i.e. learning is faster if one learns more.
登录
查看更多内容
DOI：
10.1037/h0057135
发表时间：
1952-01-01
期刊：
JOURNAL OF COMPARATIVE AND PHYSIOLOGICAL PSYCHOLOGY
影响因子：
--
作者：
LAWRENCE, DH
通讯作者：
LAWRENCE, DH
影响因子：
1.9
作者：
Izawa, J;Kondo, T;Ito, K
通讯作者：
Ito, K
影响因子：
5.3
作者：
Mandelblat-Cerf, Yael;Paz, Rony;Vaadia, Eilon
通讯作者：
Vaadia, Eilon
影响因子：
5.3
作者：
Fremaux, Nicolas;Sprekeler, Henning;Gerstner, Wulfram
通讯作者：
Gerstner, Wulfram
DOI：
10.1523/jneurosci.4284-09.2010
发表时间：
2010-06-23
期刊：
The Journal of neuroscience : the official journal of the Society for Neuroscience
影响因子：
--
作者：
Legenstein R;Chase SM;Schwartz AB;Maass W
通讯作者：
Maass W