Submitted to the Annals of Applied Probability OPTIMISTIC BAYESIAN SAMPLING IN CONTEXTUAL-BANDIT PROBLEMS ∗ By
Submitted to the Annals of Applied Probability OPTIMISTIC BAYESIAN SAMPLING IN CONTEXTUAL-BANDIT PROBLEMS ∗ By
复制标题
提交给《应用概率年鉴》 上下文强盗问题中的乐观贝叶斯采样 * 作者
DOI：
--
复制
发表时间：
2011
期刊：
影响因子：
--
通讯作者：
D. Leslie
中科院分区：
文献类型：
--
作者：
Benedict C. May;N. Korda;Anthony Lee;D. Leslie
In every sequential decision problem in an unknown environment, the decision maker faces a dilemma over whether to explore to discover more about the environment, or to exploit current knowledge. We address the exploration/exploitation dilemma in a general setting encompassing both standard and contextualised bandit problems. In this article we extend an approach of Thompson [13] which makes use of samples from the posterior distributions for the instantaneous value of each action. We also extend the approach by introducing a new algorithm, Optimistic Bayesian Sampling (OBS) in which the probability of playing an action increases with the uncertainty in the estimate of the action value. This results in better directed exploratory behaviour. We prove that, under unrestrictive assumptions, both approaches result in optimal behaviour with respect to the average reward criterion of Yang and Zhu [15].
