A study of modular models of decision making in uncertain and non-stationary environments

我们开发了统计学习模型，特别关注强化学习（RL），它可以在不确定甚至非平稳环境中进行决策。我们已经推导出一种RL方法，在该方法中，由模块结构表示的值函数可以通过以增量方式添加新模块来在线有效地近似，并且基于半参数统计框架的值函数的最优学习过程。作为一个应用，我们已经成功地在非完整系统的自动控制的基于策略的RL方法。在人类大脑中，我们发现了类似模块的结构，这些结构在推理分层推理任务时被激活。此外，我们已经成功地解码推理过程的基础上的主题的行为和MRI扫描图像。

We have developed statistical learning models, with a particular interest in reinforcement learning(RL), which can perform decision making in uncertain and even non-stationary environments. We have derived an RL method in which value function represented by a module structure can be online and efficiently approximated by adding new modules in an incremental fashion, and an optimal learning procedure of the value function based on the framework of semi-parametric statistics. As an application, we have succeeded in automatic control of non-holonomic systems by means of a policy-based RL method. In the human brain, we have found module-like structures which are activated when inferring a hierarchical inference task. Moreover, we have succeeded in decoding inference process based on the subject's behaviors and MRI scanned images.