Bias and Variance Approximation in Value Function Estimates
Bias and Variance Approximation in Value Function Estimates
复制标题
DOI：
10.1287/mnsc.1060.0614
复制
发表时间：
2007-02
期刊：
影响因子：
--
通讯作者：
Shie Mannor;D. Simester;Peng Sun;J. Tsitsiklis
中科院分区：
文献类型：
--
作者：
Shie Mannor;D. Simester;Peng Sun;J. Tsitsiklis
我们考虑有限状态、有限动作、无限范围、折扣奖励马尔可夫决策过程，并研究由模型参数的经验估计产生的价值函数估计的偏差和方差。我们提供偏差和方差的封闭形式近似值，然后可用于导出价值函数估计值的置信区间。我们使用描述邮购目录公司客户的交易和邮寄历史的大型数据库来说明和验证我们的发现。
We consider a finite-state, finite-action, infinite-horizon, discounted reward Markov decision process and study the bias and variance in the value function estimates that result from empirical estimates of the model parameters. We provide closed-form approximations for the bias and variance, which can then be used to derive confidence intervals around the value function estimates. We illustrate and validate our findings using a large database describing the transaction and mailing histories for customers of a mail-order catalog firm.