Inference for Batched Bandits
Inference for Batched Bandits
复制标题
DOI：
--
复制
发表时间：
2020-02
期刊：
影响因子：
--
通讯作者：
Kelly W. Zhang;Lucas Janson;S. Murphy
中科院分区：
文献类型：
--
作者：
Kelly W. Zhang;Lucas Janson;S. Murphy
随着强盗算法在科学研究和工业应用中的应用越来越多，对基于所产生的自适应收集的数据的可靠推理方法的需求也随之增加。在这项工作中，我们开发了使用强盗算法对批量收集的数据进行推理的方法。我们首先证明了在不存在唯一最优ARM的情况下，普通最小二乘估计量(OLS)在独立采样数据上是渐近正态的，而在用标准BANDIT算法采集的数据上不是渐近正态的。这一渐近的非正态结果意味着，假设OLS估计量近似为正态会导致第一类误差膨胀和具有低于名义覆盖概率的置信度区间。其次，我们介绍了分批OLS估计(BOLS)，我们证明了该估计(1)在多臂和背景强盗的数据上是渐近正态的，(2)对基线报酬的非平稳性是稳健的。
As bandit algorithms are increasingly utilized in scientific studies and industrial applications, there is an associated increasing need for reliable inference methods based on the resulting adaptively-collected data. In this work, we develop methods for inference on data collected in batches using a bandit algorithm. We first prove that the ordinary least squares estimator (OLS), which is asymptotically normal on independently sampled data, is not asymptotically normal on data collected using standard bandit algorithms when there is no unique optimal arm. This asymptotic non-normality result implies that the naive assumption that the OLS estimator is approximately normal can lead to Type-1 error inflation and confidence intervals with below-nominal coverage probabilities. Second, we introduce the Batched OLS estimator (BOLS) that we prove is (1) asymptotically normal on data collected from both multi-arm and contextual bandits and (2) robust to non-stationarity in the baseline reward.