When and Whom to Collaborate with in a Changing Environment: A Collaborative Dynamic Bandit Solution
When and Whom to Collaborate with in a Changing Environment: A Collaborative Dynamic Bandit Solution
复制标题
DOI：
10.1145/3404835.3462852
复制
发表时间：
2021-04
期刊：
影响因子：
--
通讯作者：
Chuanhao Li;Qingyun Wu;Hongning Wang
中科院分区：
文献类型：
--
作者：
Chuanhao Li;Qingyun Wu;Hongning Wang
协作强盗学习，即利用协作过滤技术来提高在线交互式推荐中的样本效率的强盗算法，因其两全其美而受到了广泛的研究关注。然而，所有现有的协作老虎机学习解决方案都对环境施加了静态假设，即用户偏好和用户之间的依赖关系都假设随着时间的推移是静态的。不幸的是，由于用户的兴趣和依赖关系不断变化，这种假设在实践中很难成立，这不可避免地导致推荐系统在实践中表现不佳。在这项工作中，我们开发了一种协作动态老虎机解决方案来处理不断变化的推荐环境。我们明确地将用户偏好及其依赖关系的潜在变化建模为随机过程。单个用户的偏好是通过全局共享的上下文强盗模型与狄利克雷过程先验的混合来建模的。因此，用户之间的协作是通过对全局强盗模型进行贝叶斯推理来实现的。为了平衡交互过程中的利用和探索，汤普森采样用于模型选择和臂选择。事实证明，我们的解决方案在这种充满挑战的环境中保持了标准的 ~O(√T) 贝叶斯遗憾。对合成数据集和现实世界数据集的广泛实证评估进一步证实了对不断变化的环境进行建模的必要性，以及我们的算法相对于几种最先进的在线学习解决方案的实际优势。
Collaborative bandit learning, i.e., bandit algorithms that utilize collaborative filtering techniques to improve sample efficiency in online interactive recommendation, has attracted much research attention as it enjoys the best of both worlds. However, all existing collaborative bandit learning solutions impose a stationary assumption about the environment, i.e., both user preferences and the dependency among users are assumed static over time. Unfortunately, this assumption hardly holds in practice due to users' ever-changing interests and dependency relations, which inevitably costs a recommender system sub-optimal performance in practice. In this work, we develop a collaborative dynamic bandit solution to handle a changing environment for recommendation. We explicitly model the underlying changes in both user preferences and their dependency relation as a stochastic process. Individual user's preference is modeled by a mixture of globally shared contextual bandit models with a Dirichlet process prior. Collaboration among users is thus achieved via Bayesian inference over the global bandit models. To balance exploitation and exploration during the interactions, Thompson sampling is used for both model selection and arm selection. Our solution is proved to maintain a standard ~O(√T) Bayesian regret in this challenging environment. Extensive empirical evaluations on both synthetic and real-world datasets further confirmed the necessity of modeling a changing environment and our algorithm's practical advantages against several state-of-the-art online learning solutions.