Minimizing Maximum Regret in Commitment Constrained Sequential Decision Making
Minimizing Maximum Regret in Commitment Constrained Sequential Decision Making
复制标题
最小化承诺约束的顺序决策中的最大遗憾
DOI：
10.1609/icaps.v27i1.13836
复制
发表时间：
2017
影响因子：
1.9
通讯作者：
E. Durfee
中科院分区：
文献类型：
--
作者：
Qi Zhang;Satinder Singh;E. Durfee
在多智能体合作规划中，一个智能体向其他智能体做出关于其行为方面的承诺通常是有益的，允许他们反过来规划自己的行为，而不考虑智能体的详细行为。扩展以前的工作中的贝叶斯设置，我们认为，而不是一个最坏的情况下的设置，代理有一组可能的环境（MDP），它可能是，并开发一个承诺语义，允许概率保证代理的行为在任何环境中，它可能最终面临。至关重要的是，代理接收（奖励和状态转换）的观察，使其能够潜在地消除可能的环境，从而通过使其策略适应观察的历史来获得更高的效用。我们开发了算法并提供了理论和一些初步的实证结果，表明它们确保智能体通过依赖于历史的政策履行其承诺，同时最大限度地减少对可能环境的最大遗憾。
In cooperative multiagent planning, it can often be beneficial for an agent to make commitments about aspects of its behavior to others, allowing them in turn to plan their own behaviors without taking the agent's detailed behavior into account. Extending previous work in the Bayesian setting, we consider instead a worst-case setting in which the agent has a set of possible environments (MDPs) it could be in, and develop a commitment semantics that allows for probabilistic guarantees on the agent's behavior in any of the environments it could end up facing. Crucially, an agent receives observations (of reward and state transitions) that allow it to potentially eliminate possible environments and thus obtain higher utility by adapting its policy to the history of observations. We develop algorithms and provide theory and some preliminary empirical results showing that they ensure an agent meets its commitments with history-dependent policies while minimizing maximum regret over the possible environments.