Distributionally Robust Counterpart in Markov Decision Processes
Distributionally Robust Counterpart in Markov Decision Processes
复制标题
DOI：
10.1109/tac.2015.2495174
复制
发表时间：
2015-01
影响因子：
6.8
通讯作者：
Pengqian Yu;Huan Xu
中科院分区：
文献类型：
--
作者：
Pengqian Yu;Huan Xu
本技术说明研究参数不确定性下的马尔可夫决策过程。我们采用分布鲁棒优化框架，假设不确定参数是遵循未知分布的随机变量，并寻求在最对抗的分布下最大化预期性能的策略。特别是，我们将先前的研究（集中于具有非常特殊结构的分布集）推广到更通用的分布集类别，并表明可以在温和的技术条件下有效地获得最优策略。通过以更灵活的方式合并不确定性的概率信息，显着扩展了分布式鲁棒 MDP 的适用性。
This technical note studies Markov decision processes under parameter uncertainty. We adapt the distributionally robust optimization framework, assume that the uncertain parameters are random variables following an unknown distribution, and seek the strategy which maximizes the expected performance under the most adversarial distribution. In particular, we generalize a previous study which concentrates on distribution sets with very special structure to a considerably more generic class of distribution sets, and show that the optimal strategy can be obtained efficiently under mild technical conditions. This significantly extends the applicability of distributionally robust MDPs by incorporating probabilistic information of uncertainty in a more flexible way.