Sequential Decision Making in Real-time Digital Advertising

实时竞价平台的发展改变了数字营销。这些平台允许广告商依次决定是否对特定的广告位置（如网站）进行投标，投标多少以及向谁投标。市场的速度意味着这些决策需要几乎在瞬间做出，这导致了许多算法方法的使用。数字广告领域决策的另一个关键方面涉及隐私和效用之间的平衡。对于营销人员来说，理想的情况是能够访问用户的整个在线浏览历史，而对于互联网用户来说，理想的设置是营销人员能够最小限度地访问他们的信息。这就导致了一种权衡，即用户所能负担得起的最相关的浏览体验与他们愿意与营销人员分享的信息之间的权衡。实时竞价问题的细微差别意味着直接应用现有算法进行顺序决策可能会导致做出次优决策。因此，该项目将涉及设计和分析针对数字广告特定设置的新算法，包括隐私保护学习。由于b谷歌公司弃用了“cookie”（最常用的用于跟踪用户在线活动的数字标识符），这一点最近变得极其重要，几乎对许多广告技术公司构成了生存威胁。在没有标识符的情况下，如何为互联网用户提供个性化的隐私保护浏览体验？在这个项目中，我们的目标是设计新的算法，从强化学习算法（处理具有不确定性的顺序决策）、半监督学习算法（使用聚合信息或部分标记数据来学习和优化策略）和主动学习方法（专注于如何在有限样本下有效学习，特别是当标记数据昂贵时）中汲取灵感。该项目属于EPSRC的研究主题“运筹学”、“统计与应用概率”和“数字经济”，并与EPSRC的战略重点“人工智能、数字化和数据：驱动价值和安全”保持一致。这项研究是与“伦敦帝国理工学院”和“The Trade Desk”合作完成的，后者是一家专门从事实时程序化营销自动化技术的跨国公司。为此，我们的目标是双重的：a)设计严格的算法，并提供可证明的保证；b)证明我们的算法对真实数据的有效性，并推动对现实世界的影响。

Digital marketing has been transformed by the development of real-time bidding platforms. These platforms allow advertisers to decide sequentially whether to place a bid on a particular advertising location (e.g., a website), how much to bid on and who to bid on. The speed of the markets means that these decisions need to be taken almost instantaneously, which has led to many algorithmic approaches being used. Another key aspect of decision-making in the digital advertising world involves the balance between privacy and utility. The ideal situation for the marketer is to have access to the user's entire online browsing history whereas the ideal set up for an internet user is for the marketers to have minimal access to their information. This leads to a trade-off between what is the most affordable and possibly relevant browsing experience a user can have with the information they are willing to share with the marketers. The nuances of the real-time bidding problem mean that directly applying existing algorithms for sequential decision making may lead to sub-optimal decisions being taken. Therefore, the project will involve designing and analysing new algorithms tailored to specific settings in digital advertising, including privacy preserving learning. This has become extremely important and almost an existential threat for many advertising technology companies in recent time due to the deprecation of "cookies" (most popularly used digital identifier used to track user's online activity) by Google Inc. Without access to an identifier, how does one deliver a personalized privacy preserving browsing experience for an internet user? In this project, we aim to design new algorithms that draw ideas from reinforcement learning algorithms that deal with sequential decision-making with uncertainty, semi supervised learning algorithms that use aggregate information or partially labelled data to learn and optimize policies and active learning methodologies that focus on how to learn efficiently with limited samples, particularly when labelling data is expensive. This project falls within the EPSRC research themes of "Operational Research", "Statistics and Applied Probability" and "Digital economy" and aligns with EPSRC's strategic priority related to "artificial intelligence, digitisation and data: driving value and security". This research is being done in collaboration with "Imperial College, London" and "The Trade Desk", which is a multinational company that specializes in real-time programmatic marketing automation technologies. Towards this, our goals are two-fold: a) Design rigorous algorithms with provable guarantees b) Demonstrate the efficacy of our algorithms on real data and drive towards real world impact.