Low-Rank Generalized Linear Bandit Problems
Low-Rank Generalized Linear Bandit Problems
复制标题
低阶广义线性老虎机问题
DOI：
--
复制
发表时间：
2020
期刊：
影响因子：
--
通讯作者：
Ambuj Tewari
中科院分区：
文献类型：
--
作者：
Yangyi Lu;A. Meisami;Ambuj Tewari
在低秩线性盗贼问题中，一个动作(由一个大小为$d1×d2$的矩阵表示)的报酬是该动作与一个未知的低阶矩阵$Theta^*$之间的内积。我们提出了一种基于在线到置信度集转换的新算法-CITEP{abbasi2012Online}和由低阶矩阵覆盖构造的指数加权平均预测器。在$T$轮中，我们的算法得到$widetilde{O}((d_1+d_2)^{3/2}sqrt{rt})$遗憾，当$theta^*$：$rll min{d_1，d_2}$时，改进了$widetilde{O}(d_1d_2sqrt{T})$的标准线性盗贼遗憾界.我们还将我们的算法方法扩展到广义线性设置，得到了一个在链接函数的正则性条件下具有类似上界的算法。为了克服基于覆盖的方法的计算困难，我们提出了一种有效的算法，该算法扩展了CITET{ju2019billinine}的“探索-子空间-然后求精”算法。我们的高效算法在动作集$mathcal{X}$和$r$次奇异值$Theta^*$的温和条件下实现了$宽{O}((d_1+d_2)^{3/2}SQRT})$遗憾。我们的上界与一类低阶线性盗贼问题的猜测下界相匹配。进一步，我们证明了稀疏线性盗贼问题的现有下界强烈地表明我们的遗憾界是不可改进的。为了补充我们的理论贡献，我们还进行了实验，证明了当$Theta^*$是低阶时，我们的算法可以大大优于标准的线性盗贼方法。
In a low-rank linear bandit problem, the reward of an action (represented by a matrix of size $d_1 imes d_2$) is the inner product between the action and an unknown low-rank matrix $Theta^*$. We propose an algorithm based on a novel combination of online-to-confidence-set conversion~citep{abbasi2012online} and the exponentially weighted average forecaster constructed by a covering of low-rank matrices. In $T$ rounds, our algorithm achieves $widetilde{O}((d_1+d_2)^{3/2}sqrt{rT})$ regret that improves upon the standard linear bandit regret bound of $widetilde{O}(d_1d_2sqrt{T})$ when the rank of $Theta^*$: $r ll min{d_1,d_2}$. We also extend our algorithmic approach to the generalized linear setting to get an algorithm which enjoys a similar bound under regularity conditions on the link function. To get around the computational intractability of covering based approaches, we propose an efficient algorithm by extending the "Explore-Subspace-Then-Refine" algorithm of~citet{jun2019bilinear}. Our efficient algorithm achieves $widetilde{O}((d_1+d_2)^{3/2}sqrt{rT})$ regret under a mild condition on the action set $mathcal{X}$ and the $r$-th singular value of $Theta^*$. Our upper bounds match the conjectured lower bound of cite{jun2019bilinear} for a subclass of low-rank linear bandit problems. Further, we show that existing lower bounds for the sparse linear bandit problem strongly suggest that our regret bounds are unimprovable. To complement our theoretical contributions, we also conduct experiments to demonstrate that our algorithm can greatly outperform the performance of the standard linear bandit approach when $Theta^*$ is low-rank.
影响因子：
6.3
作者：
Fan, Jianqing;Gong, Wenyan;Zhu, Ziwei
通讯作者：
Zhu, Ziwei