Multi-facet Contextual Bandits: A Neural Network Perspective
Multi-facet Contextual Bandits: A Neural Network Perspective
复制标题
DOI：
10.1145/3447548.3467299
复制
发表时间：
2021-06
期刊：
影响因子：
--
通讯作者：
Yikun Ban;Jingrui He;C. Cook
中科院分区：
文献类型：
--
作者：
Yikun Ban;Jingrui He;C. Cook
情境多臂老虎机已被证明是推荐系统中的一种有效工具。在本文中，我们研究了一个新的多面老虎机问题，它涉及一组老虎机，每个老虎机从一个独特的方面描述用户的需求。在每一轮中，对于给定的用户，我们需要从每个老虎机中选择一个臂，使得所有臂的组合能使最终奖励最大化。这个问题可以立即在电子商务、医疗保健等领域得到应用。为了解决这个问题，我们提出了一种新的算法，名为MuFasa，它利用一个组合神经网络来联合学习多个老虎机的潜在奖励函数。它估计一个与预期奖励相关的上置信界（UCB），以在利用和探索之间取得平衡。在温和的假设下，我们对MuFasa进行了遗憾分析。它可以实现接近最优的$\tilde{O}((K + 1)\sqrt{T})$遗憾界，其中K是老虎机的数量，T是进行的轮数。此外，我们进行了大量的实验，以表明MuFasa在真实数据集上优于强大的基线方法。
Contextual multi-armed bandit has shown to be an effective tool in recommender systems. In this paper, we study a novel problem of multi-facet bandits involving a group of bandits, each characterizing the users' needs from one unique aspect. In each round, for the given user, we need to select one arm from each bandit, such that the combination of all arms maximizes the final reward. This problem can find immediate applications in E-commerce, healthcare, etc. To address this problem, we propose a novel algorithm, named MuFasa, which utilizes an assembled neural network to jointly learn the underlying reward functions of multiple bandits. It estimates an Upper Confidence Bound (UCB) linked with the expected reward to balance between exploitation and exploration. Under mild assumptions, we provide the regret analysis of MuFasa. It can achieve the near-optimal Õ((K + 1) √T) regret bound where K is the number of bandits and T is the number of played rounds. Furthermore, we conduct extensive experiments to show that MuFasa outperforms strong baselines on real-world data sets.