What recommendation algorithms are optimal in (MAB) settings when they compete, and evaluates the intertemporal welfare effect of competition

当Trivago这样的推荐服务竞争时，哪些消费者会受益？参赛者应该选择什么推荐策略？这项研究项目探索了在多臂土匪(MAB)环境下，当他们竞争时，哪些推荐算法是最优的，并评估了竞争的跨期福利效应。MAB是一种流行的工具，用于建模从医学试验到互联网经济的探索-开发权衡。推荐者试图通过说服消费者选择想要的商品来收集不同产品的回报分布信息。当推荐者单独行动时，例如谷歌的广告收入最大化算法，这个问题就得到了很好的研究。然而，很少有人关注代理面临来自多个算法的建议的情况。换言之，现有文献探讨了当顾客在产品之间进行选择而不是产品-推荐者配对时的MAB问题.当算法竞争客户时，两个效应影响消费者效用：(I)对于给定的算法集，由于每个算法都观察到可用信息的严格子集，所以信息获取较慢；(Ii)在没有竞争的情况下可能会选择不同的算法。更智能的算法将减少持续的次优选择带来的累积后悔，但由于信息获取，对较早的客户的预期效用较低。它们的计算成本也更高。所有这些都对消费者效用的跨期分布提出了有趣的问题。我的研究问题与最近的两篇论文直接相关。在《竞争强盗：在竞争中学习》一书中，曼苏尔等人写道。(2018)在非常有限的临时假设下推导出一些解析解。Aridor等人。(2019)考虑更自然的设置，但在《竞争下的探索的危险：计算建模方法》中使用模拟来近似最优算法。他们的模型的一个共同特点是，客户只收到一种推荐。在互联网经济中，查看多个网站实际上是没有成本的，为了建立模型，我转而假设每个消费者都遵循每个算法的推荐，然后选择一种产品。竞争的产生是因为消费者只向一种算法透露他们的体验。我计划解析地求解最优算法。哪种算法是最优的将在很大程度上取决于客户知道什么。在Mansour等人的文章中。(2018)设置，基本的贪婪算法获胜，但在Aridor等人。(2019)，在足够长的时间里，更复杂的算法击败了贪婪的算法。同样，对委托人可以观察到的信息的假设也极其重要。委托人观察到的越多，我预计最优算法就越贪婪，处于任何非合作均衡状态。我计划在适当的时候用模拟来补充理论发现。这项研究对在线市场的政策制定者和监管机构具有重要意义。

Which consumers gain when recommendation services like Trivago compete? What recommendation strategy should an entrant pick? This research project explores what recommendation algorithms are optimal in multi-armed bandit (MAB) settings when they compete, and evaluates the intertemporal welfare effect of competition.MABs are a popular tool to model explore-exploit trade-offs - from medical trials to the internet economy. A recommender tries to gather information on payoff distributions of different products by persuading consumers to choose the desired good. When a recommender acts in isolation, for example, Google's ad-revenue-maximization algorithm, the problem is well studied. However, little attention has been given to situations where agents are faced with recommendations from multiple algorithms. In other words, the existing literature has explored the MAB problem when customers choose between products rather than product-recommender pairs.When algorithms compete for customers, two effects impact consumer utility, (i) for a given set of algorithms, information acquisition is slower as each algorithm observes a strict subset of the available information, and (ii) different algorithms may be chosen in case of no competition. Smarter algorithms will reduce cumulative regret from persistent suboptimal choices but have a lower expected utility for earlier customers due to information acquisition. They are also computationally costlier. All of the above raises interesting questions about the intertemporal distribution of consumer utility. My research question relates directly to two recent papers. In 'Competing Bandits: Learning Under Competition', Mansour et al. (2018) derive some analytical solutions under very restrictive ad hoc assumptions. Aridor et al. (2019) consider a more natural setting but use simulations to approximate the optimal algorithm in 'The Perils of Exploration under Competition: A Computational Modeling Approach'. A common feature of their models is that customers only receive one recommendation. To model setups like the internet economy, where checking multiple websites is effectively costless, I instead assume that each consumer observes a recommendation from each algorithm and chooses a product thereafter. Competition arises because the consumer only reveals their experience to one algorithm.I plan to solve for the optimal algorithm analytically. Which algorithm is optimal will depend heavily on what customers know. In the Mansour et al. (2018) setup, the basic greedy algorithm wins, but in Aridor et al. (2019), for long enough horizons, more sophisticated algorithms beat the greedy one. Similarly, assumptions on the information principals can observe are extremely important. The more principals can observe, the greedier I expect the optimal algorithm to be in any non-cooperative equilibrium. I plan to supplement the theoretical findings with simulations where appropriate. The research has important implications for policymakers and regulators of online markets.