Bandits with Delayed, Aggregated Anonymous Feedback
Bandits with Delayed, Aggregated Anonymous Feedback
复制标题
DOI：
--
复制
发表时间：
2017-09
期刊：
影响因子：
--
通讯作者：
Ciara Pike-Burke;Shipra Agrawal;Csaba Szepesvari;S. Grünewälder
中科院分区：
文献类型：
--
作者：
Ciara Pike-Burke;Shipra Agrawal;Csaba Szepesvari;S. Grünewälder
我们研究的随机K-臂土匪问题，我们称之为“土匪延迟，聚合匿名反馈”的一个变种。在这个问题中，当玩家拉动手臂时，奖励会产生，但不会立即观察到。相反，在每一轮结束时，玩家只观察到在给定回合中碰巧到达的先前生成的奖励的数量的总和。奖励是随机延迟的，由于观察的综合性质，导致特定奖励的手臂的信息丢失。问题是，由于这种延迟的、聚合的匿名反馈，信息丢失的成本是多少？以前的作品研究了土匪随机，非匿名的延迟，并发现，遗憾的增加，只有一个附加因素有关的预期延迟。在本文中，我们表明，这种添加剂的遗憾增加可以保持在较硬的延迟，聚合匿名反馈设置时，预期的延迟（或它的约束）是已知的。我们提供了一个算法，匹配的最坏情况下的遗憾的非匿名问题时，延迟是有界的，和对数因子或一个附加的方差项无界延迟。
We study a variant of the stochastic K-armed bandit problem, which we call “bandits with delayed, aggregated anonymous feedback”. In this problem, when the player pulls an arm, a reward is generated, however it is not immediately observed. Instead, at the end of each round the player observes only the sum of a number of previously generated rewards which happen to arrive in the given round. The rewards are stochastically delayed and due to the aggregated nature of the observations, the information of which arm led to a particular reward is lost. The question is what is the cost of the information loss due to this delayed, aggregated anonymous feedback? Previous works have studied bandits with stochastic, non-anonymous delays and found that the regret increases only by an additive factor relating to the expected delay. In this paper, we show that this additive regret increase can be maintained in the harder delayed, aggregated anonymous feedback setting when the expected delay (or a bound on it) is known. We provide an algorithm that matches the worst case regret of the non-anonymous problem exactly when the delays are bounded, and up to logarithmic factors or an additive variance term for unbounded delays.