Model-based reinforcement learning : brain implementation and engineering applications

[在线贝叶斯学习方案]我们设计了一种在线贝叶斯学习算法，该算法可以应用于高斯随机过程，并可以估计系统的维度和目标动态中的变化发生(Hirayama等人，2004)。我们还设计了一种可应用于非高斯随机过程的序贯蒙特卡罗方法，并将其应用于视觉跟踪问题(Bando等人，在出版社)。[基于模型的强化学习和在线学习的应用]我们成功地基于中心模式生成器和强化学习的组合，允许两足机器人模拟器自主行走。我们后来对这种方法进行了扩展，将基于策略梯度的强化学习纳入其中。通过进一步引入在线模型辨识方法，两足动物模拟器的自主学习得到了加速(Nakamura等人，2005年)。我们对开关控制器的强化学习成功地启动了一个…更多的是稳定一个驱动不足的真实机器人，即杂技机器人。基于模型强化学习和在线模型学习相结合的自主训练方案可以为多智能体纸牌游戏构建一个与人类专家玩家一样强大的纸牌游戏代理(Ishii，et al.，2005)。[灵长类动物与奖赏相关的前额神经活动]一项基于灵长类动物记忆的感觉运动加工任务的电生理学研究表明，奖赏期望显著提高了感觉工作记忆的选择性，但不能提高运动记忆的选择性(Aemeri等人，2005)。[人类前额信息处理的神经心理学研究]我们建立了一个人类执行马尔科夫决策过程的信息处理模型，并通过功能磁共振成像进行神经心理学研究，对模型的合理性进行评价。我们发现了背外侧前额叶皮质的参与(Yoshida，et al.，2005)。当马尔可夫决策环境涉及不确定性时，其解析可以在前极前额叶皮质执行(Yoshida等人，在出版社)。较少

[On-line Bayesian learning schemes]We devised an on-line Bayesian learning algorithm which can be applied to Gaussian stochastic processes and can estimate the system dimensionality and change occurrence in the target dynamics (Hirayama et al., 2004). We also devised a sequential Monte-Carlo-based method which can be applied to non-Gaussian stochastic processes and applied it to visual tracking problems (Bando, et al., in press).[Applications of model-based reinforcement learning and on-line learning]We succeeded in allowing a biped robot simulator to biped-walk autonomously, based on the combination of central pattern generator and reinforcement learning. We later extended this approach such to incorporate policy-gradient-based reinforcement learning. By further introducing an on-line model identification method, the autonomous learning by the biped simulator has been accelerated (Nakamura et al., 2005). Our reinforcement learning for a switching controller succeeded in swinging-up an … More d stabilizing an underactuated real robot, the acrobot. An autonomous training scheme based on the combination of the model-based reinforcement learning and the on-line model learning can construct a card-game playing agent for a multi-agent card game, which is as strong as a human expert player (Ishii, et al., 2005).[Reward-related prefrontal neural activities of primates]An electrophysiological study with a primates memory-based sensorimotor processing task revealed that the reward expectation significantly enhanced the selectivity of sensory working memory but not that of motor memory (Amemori, et al., 2005).[Neuropsychological study of humans prefrontal information processing]We developed an information processing model during a human performs a Markov decision process, and evaluated the model plausibility by means of neuropsychological studies with functional magnetic resonance imaging. We found the engagement of dorsolateral prefrontal cortex (Yoshida, et al., 2005). When the Markov decision environment involves uncertainty, its resolution could be performed in front-polar prefrontal cortex (Yoshida, et al., in press). Less