Delay and Cooperation in Nonstochastic Linear Bandits
Delay and Cooperation in Nonstochastic Linear Bandits
复制标题
非随机线性强盗中的延迟与合作
DOI：
10.1051/0004-6361/202345845
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
K. Kawarabayashi
中科院分区：
文献类型：
--
作者：
Shinji Ito;Daisuke Hatano;Hanna Sumita;Kei Takemura;Takuro Fukunaga;Naonori Kakimura;K. Kawarabayashi
本文提出了一种带延迟反馈的在线线性优化问题的近似最优算法。在线线性优化与强盗反馈，或非随机线性强盗，提供了一个通用的框架，顺序决策问题的信息有限。然而，这个框架假设反馈可以在选择动作后立即观察到，因此，不直接适用于许多实际应用，其中反馈通常只能在一段时间后才能获得。为了科普这种情况下，我们考虑的问题设置，其中的反馈可以观察到d轮后，选择的行动，并提出了一个算法，其预期的遗憾是O（p m（m + d）T），忽略对数因子的m和T，其中m和T表示的行动集的维数和轮数，分别。该算法实现了接近最优的性能，因为我们能够表明，任意算法遭受的遗憾，在最坏的情况下，（p m（m + d）T）。为了开发该算法，我们引入了一种技术，我们称之为分布截断，它在约束遗憾中起着至关重要的作用。我们还将我们的方法应用于合作土匪，如Cesa-Bianchi等人所研究的。[18]和Bar-On和Mansour [12]，并将他们的结果扩展到线性土匪设置。
This paper offers a nearly optimal algorithm for online linear optimization with delayed bandit feedback. Online linear optimization with bandit feedback, or nonstochastic linear bandits, provides a generic framework for sequential decision-making problems with limited information. This framework, however, assumes that feedback can be observed just after choosing the action, and, hence, does not apply directly to many practical applications, in which the feedback can often only be obtained after a while. To cope with such situations, we consider problem settings in which the feedback can be observed d rounds after the choice of an action, and propose an algorithm for which the expected regret is ˜ O ( p m ( m + d ) T ) , ignoring logarithmic factors in m and T , where m and T denote the dimensionality of the action set and the number of rounds, respectively. This algorithm achieves nearly optimal performance, as we are able to show that arbitrary algorithms suffer the regret of ⌦ ( p m ( m + d ) T ) in the worst case. To develop the algorithm, we introduce a technique we refer to as distribution truncation , which plays an essential role in bounding the regret. We also apply our approach to cooperative bandits, as studied by Cesa-Bianchi et al. [18] and Bar-On and Mansour [12], and extend their results to the linear bandits setting.
多代理多武装强盗中的社会学习
DOI：
10.1145/3393691.3394217
发表时间：
2019
期刊：
Proceedings of the ACM on Measurement and Analysis of Computing Systems
影响因子：
--
作者：
Sankararaman, Abishek;Ganesh, Ayalvadi;Shakkottai, Sanjay
通讯作者：
Shakkottai, Sanjay