A Fenchel-Moreau-Rockafellar type theorem on the Kantorovich-Wasserstein space with applications in partially observable Markov decision processes
A Fenchel-Moreau-Rockafellar type theorem on the Kantorovich-Wasserstein space with applications in partially observable Markov decision processes
复制标题
Kantorovich-Wasserstein 空间上的 Fenchel-Moreau-Rockafellar 型定理及其在部分可观测马尔可夫决策过程中的应用
DOI：
10.1016/j.jmaa.2019.05.004
复制
发表时间：
2019
影响因子：
1.3
通讯作者：
Wilhelm Stannat
中科院分区：
文献类型：
--
作者：
Vaios Laschos;Klaus Obermayer;Yun Shen;Wilhelm Stannat
利用具有有限支集的所有概率测度空间可以以两种不同的方式完成，一种是生成Arens-Eells空间，另一种是生成Kantorovich-Wasserstein(Wasserstein-1)空间，并利用Arens-Eells空间与Lipschitz函数空间之间的对偶关系，给出了Wasserstein-1上真凸泛函的Fancel-Moreau-Rockafelel型的对偶表示。作为推论，我们检索了对偶运输不等式，并提供了一些例子，其中该定理可以很容易地证明像著名的Donsker-Varadhan变分公式那样的对偶表达式。最后，我们的结果允许将凸函数写为由其共轭对偶的根生成的所有线性函数的上确界，我们将其应用于部分可观测马尔可夫决策过程(POMDP)领域，以通过迭代水平集来逼近给定POMDP的值函数。这将Smallwood和Sondik(1973)[20]中用于有限状态空间的方法推广到状态空间是Polish度量空间的情况。
By using the fact that the space of all probability measures with finite support can be completed in two different fashions, one generating the Arens-Eells space and another generating the Kantorovich-Wasserstein (Wasserstein-1) space, and by exploiting the duality relationship between the Arens-Eells space with the space of Lipschitz functions, we provide a dual representation of Fenchel-Moreau-Rockafellar type for proper convex functionals on Wasserstein-1. We retrieve dual transportation inequalities as a Corollary and we provide examples where the theorem can be used to easily prove dual expressions like the celebrated Donsker-Varadhan variational formula. Finally our result allows to write convex functions as the supremum over all linear functions that are generated by roots of its conjugate dual, something that we apply to the field of Partially observable Markov decision processes (POMDPs) to approximate the value function of a given POMDP by iterating level sets. This extends the method used in Smallwood and Sondik (1973) [20] for finite state spaces to the case were the state space is a Polish metric space.