Learning Neural Contextual Bandits through Perturbed Rewards-猫眼课题宝

DOI：

发表时间：

2022-01

期刊：

影响因子：

通讯作者：

中科院分区：

其他

文献类型：

作者：

Yiling Jia;Weitong Zhang;Dongruo Zhou;Quanquan Gu;Hongning Wang

由于表征学习的力量，神经上下文强盗算法与经典算法相比表现出显着的性能改进。但由于他们的探索必须在整个神经网络参数空间中进行，以获得接近最优的遗憾，因此计算成本非常高。我们在更新神经网络时对奖励进行扰动，以消除显式探索的需要和相应的计算开销。我们证明了在标准正则性条件下仍然可以实现$\tilde{O}(\tilde{d}\sqrt{T})$遗憾上界，其中$T$是相互作用的轮数，$\tilde{d}$是神经切线核矩阵的有效维数。与几种基准上下文强盗算法（包括两种最近的神经上下文强盗模型）进行了广泛的比较，证明了我们提出的神经强盗算法的有效性和计算效率。

Thanks to the power of representation learning, neural contextual bandit algorithms demonstrate remarkable performance improvement against their classical counterparts. But because their exploration has to be performed in the entire neural network parameter space to obtain nearly optimal regret, the resulting computational cost is prohibitively high. We perturb the rewards when updating the neural network to eliminate the need of explicit exploration and the corresponding computational overhead. We prove that a $\tilde{O}(\tilde{d}\sqrt{T})$ regret upper bound is still achievable under standard regularity conditions, where $T$ is the number of rounds of interactions and $\tilde{d}$ is the effective dimension of a neural tangent kernel matrix. Extensive comparisons with several benchmark contextual bandit algorithms, including two recent neural contextual bandit models, demonstrate the effectiveness and computational efficiency of our proposed neural bandit algorithm.

Learning Neural Contextual Bandits through Perturbed Rewards

文献摘要

相似文献