Discovering latent causes in reinforcement learning
Discovering latent causes in reinforcement learning
复制标题
DOI：
10.1016/j.cobeha.2015.07.007
复制
发表时间：
2015-10-01
影响因子：
5
通讯作者：
Niv, Yael
中科院分区：
文献类型：
--
作者：
Gershman, Samuel J.;Norman, Kenneth A.;Niv, Yael
有效的强化学习取决于拥有适当的状态表示。但这种表征从何而来？我们认为，大脑通过尝试推断手头任务的潜在因果结构并将每个潜在原因分配给一个单独的状态来发现状态表征。在本文中，我们回顾了这个潜在原因框架的几个含义，重点是巴甫洛夫条件反射。该框架表明，条件反射不是获得线索和结果之间的关联，而是获得潜在原因和可观察刺激之间的关联。对条件反射的潜在原因解释使我们能够开始回答那些令经典理论受挫的问题：为什么消失的反应有时会回来？为什么以复合形式呈现的刺激有时会相加，有时则不会？除了条件作用之外，潜在因果推理的原理还可以提供跨认知领域的结构学习的一般理论。
Effective reinforcement learning hinges on having an appropriate state representation. But where does this representation come from? We argue that the brain discovers state representations by trying to infer the latent causal structure of the task at hand, and assigning each latent cause to a separate state. In this paper, we review several implications of this latent cause framework, with a focus on Pavlovian conditioning. The framework suggests that conditioning is not the acquisition of associations between cues and outcomes, but rather the acquisition of associations between latent causes and observable stimuli. A latent cause interpretation of conditioning enables us to begin answering questions that have frustrated classical theories: Why do extinguished responses sometimes return? Why do stimuli presented in compound sometimes summate and sometimes do not? Beyond conditioning, the principles of latent causal inference may provide a general theory of structure learning across cognitive domains.