Structural Estimation of Partially Observable Markov Decision Processes
Structural Estimation of Partially Observable Markov Decision Processes
复制标题
DOI：
10.1109/tac.2022.3217908
复制
发表时间：
2020-08
影响因子：
6.8
通讯作者：
Yanling Chang;Alfredo Garcia;Zhide Wang;Lu Sun
中科院分区：
文献类型：
--
作者：
Yanling Chang;Alfredo Garcia;Zhide Wang;Lu Sun
部分可观测马尔可夫决策过程（POMDPs）是不确定性和部分信息下序贯决策的一个成熟框架。本文认为（逆）结构估计的POMDP基元的基础上的数据序列的形式的可观的和实施的行动。我们分析的熵正则化POMDP的结构特性和指定的条件下，该模型是可识别的状态动力学的知识。我们考虑一个软政策梯度算法来计算最大似然估计，并说明了一个设备更换问题的方法。
Partially observable Markov decision processes (POMDPs) is a well-developed framework for sequential decision-making under uncertainty and partial information. This article considers the (inverse) structural estimation of the primitives of a POMDP based upon data in the form of sequences of observables and implemented actions. We analyze the structural properties of an entropy regularized POMDP and specify conditions under which the model is identifiable without knowledge of the state dynamics. We consider a soft policy gradient algorithm to compute a maximum likelihood estimator, and illustrate the approach with an equipment replacement problem.