Sequential Decision Making in Real-time Digital Advertising

实时竞价平台的发展使数字营销发生了转变。这些平台允许广告商顺序地决定是否在特定广告位置(例如，网站)上出价、出价多少以及出价对象。市场的速度意味着，这些决定几乎需要立即做出，这导致了许多算法方法的使用。数字广告界决策的另一个关键方面涉及隐私和效用之间的平衡。对于营销者来说，理想的情况是能够访问用户的整个在线浏览历史，而对于互联网用户来说，理想的设置是营销者对他们的信息的访问权限最小。这导致在用户可以拥有的最实惠和可能相关的浏览体验与他们愿意与营销人员分享的信息之间进行权衡。实时竞价问题的细微差别意味着，直接应用现有算法进行顺序决策可能会导致做出次优决策。因此，该项目将涉及设计和分析为数字广告中的特定环境量身定做的新算法，包括隐私保护学习。最近，由于谷歌公司不再推荐Cookie(最常用的数字标识，用于跟踪用户的在线活动)，这对许多广告技术公司来说已经变得极其重要，几乎是一个生存威胁。如果没有标识，人们如何为互联网用户提供个性化的隐私保护浏览体验？在这个项目中，我们的目标是设计新的算法，从处理不确定性序列决策的强化学习算法，使用聚合信息或部分标记数据来学习和优化策略的半监督学习算法，以及专注于如何在有限样本下有效学习的主动学习方法，特别是在标记数据昂贵的情况下。该项目属于EPSRC“运筹学”、“统计与应用概率”和“数字经济”的研究主题，并与EPSRC与“人工智能、数字化和数据：推动价值和安全”相关的战略优先事项保持一致。这项研究是与“伦敦帝国理工学院”和“The Trade Desk”合作进行的，后者是一家专门从事实时程序化营销自动化技术的跨国公司。为此，我们的目标有两个：a)设计严格的算法，并提供可证明的保证；b)展示我们的算法在真实数据上的有效性，并推动实现真实世界的影响。

Digital marketing has been transformed by the development of real-time bidding platforms. These platforms allow advertisers to decide sequentially whether to place a bid on a particular advertising location (e.g., a website), how much to bid on and who to bid on. The speed of the markets means that these decisions need to be taken almost instantaneously, which has led to many algorithmic approaches being used. Another key aspect of decision-making in the digital advertising world involves the balance between privacy and utility. The ideal situation for the marketer is to have access to the user's entire online browsing history whereas the ideal set up for an internet user is for the marketers to have minimal access to their information. This leads to a trade-off between what is the most affordable and possibly relevant browsing experience a user can have with the information they are willing to share with the marketers. The nuances of the real-time bidding problem mean that directly applying existing algorithms for sequential decision making may lead to sub-optimal decisions being taken. Therefore, the project will involve designing and analysing new algorithms tailored to specific settings in digital advertising, including privacy preserving learning. This has become extremely important and almost an existential threat for many advertising technology companies in recent time due to the deprecation of "cookies" (most popularly used digital identifier used to track user's online activity) by Google Inc. Without access to an identifier, how does one deliver a personalized privacy preserving browsing experience for an internet user? In this project, we aim to design new algorithms that draw ideas from reinforcement learning algorithms that deal with sequential decision-making with uncertainty, semi supervised learning algorithms that use aggregate information or partially labelled data to learn and optimize policies and active learning methodologies that focus on how to learn efficiently with limited samples, particularly when labelling data is expensive. This project falls within the EPSRC research themes of "Operational Research", "Statistics and Applied Probability" and "Digital economy" and aligns with EPSRC's strategic priority related to "artificial intelligence, digitisation and data: driving value and security". This research is being done in collaboration with "Imperial College, London" and "The Trade Desk", which is a multinational company that specializes in real-time programmatic marketing automation technologies. Towards this, our goals are two-fold: a) Design rigorous algorithms with provable guarantees b) Demonstrate the efficacy of our algorithms on real data and drive towards real world impact.