Credit assignment in multiple goal embodied visuomotor behavior.
Credit assignment in multiple goal embodied visuomotor behavior.
复制标题
DOI：
10.3389/fpsyg.2010.00173
复制
发表时间：
2010
影响因子：
3.8
通讯作者：
Ballard DH
中科院分区：
文献类型：
--
作者：
Rothkopf CA;Ballard DH
大脑的内在复杂性可能会导致人们将其与身体关系的问题放在一边，但具身认知领域强调，在系统水平上理解大脑功能需要解决脑-体界面的作用。直到最近人们才意识到，这个界面执行了大量的计算，而不必由大脑重复，因此在其表示中为大脑提供了极大的简化。实际上，大脑的抽象状态可以指身体创造的世界的编码表示。但是，即使大脑可以通过抽象与世界交流，其神经回路中严重的速度限制意味着在发育期间必须执行大量的索引，以便能够快速获得适当的行为反应。一种可能的方式是，如果大脑使用分解，从而可以快速访问和组合行为原语。这种认识促使我们研究独立的感觉运动任务解决者，我们称之为模块，在指导行为。我们在这里关注的问题是一个具体的代理如何学习校准这样的个人视觉模块，同时追求多个目标。模块编程的生物学上合理的标准是在探索环境的过程中给予强化。然而，当感觉运动模块组合使用时，该公式包含一个实质性问题：它们的整体性能的功劳必须在它们之间分配。我们表明，这个问题是可以解决的，不同的任务组合是有益的学习，而不是一个复杂的，通常假设。我们的模拟表明，快速算法是正确分配信用，是不敏感的测量噪声。
The intrinsic complexity of the brain can lead one to set aside issues related to its relationships with the body, but the field of embodied cognition emphasizes that understanding brain function at the system level requires one to address the role of the brain-body interface. It has only recently been appreciated that this interface performs huge amounts of computation that does not have to be repeated by the brain, and thus affords the brain great simplifications in its representations. In effect the brain's abstract states can refer to coded representations of the world created by the body. But even if the brain can communicate with the world through abstractions, the severe speed limitations in its neural circuitry mean that vast amounts of indexing must be performed during development so that appropriate behavioral responses can be rapidly accessed. One way this could happen would be if the brain used a decomposition whereby behavioral primitives could be quickly accessed and combined. This realization motivates our study of independent sensorimotor task solvers, which we call modules, in directing behavior. The issue we focus on herein is how an embodied agent can learn to calibrate such individual visuomotor modules while pursuing multiple goals. The biologically plausible standard for module programming is that of reinforcement given during exploration of the environment. However this formulation contains a substantial issue when sensorimotor modules are used in combination: The credit for their overall performance must be divided amongst them. We show that this problem can be solved and that diverse task combinations are beneficial in learning and not a complication, as usually assumed. Our simulations show that fast algorithms are available that allot credit correctly and are insensitive to measurement noise.
登录
查看更多内容
影响因子：
5.7
作者：
Hikosaka, Okihide;Bromberg-Martin, Ethan;Hong, Simon;Matsumoto, Masayuki
通讯作者：
Matsumoto, Masayuki
影响因子：
64.8
作者：
Daw, Nathaniel D.;O'Doherty, John P.;Dayan, Peter;Seymour, Ben;Dolan, Raymond J.
通讯作者：
Dolan, Raymond J.
影响因子：
2.9
作者：
Doya, K;Samejima, K;Kawato, M
通讯作者：
Kawato, M
DOI：
10.1080/095281397147103
发表时间：
1997-04-01
影响因子：
2.2
作者：
Bonasso, RP;Firby, RJ;Slack, MG
通讯作者：
Slack, MG
影响因子：
14.4
作者：
Boutilier, C;Dearden, R;Goldszmidt, M
通讯作者：
Goldszmidt, M