RI: Small: Collaborative Research: RUI: Batch Learning from Logged Bandit Feedback

RI：小型：协作研究：RUI：从记录的强盗反馈中批量学习

• 批准号: 1615679
• 负责人: Douglas Turnbull
• 金额: $ 10万
• 依托单位: Ithaca College
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-07-01 至 2020-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1615679&HistoricalAwards=false
• 关键词: RI; Small; Collaborative; Research; RUI

Log data is one of the most ubiquitous forms of data available, as it can be recorded from a variety of systems (e.g., search engines, recommender systems, ad placement platforms) at little cost. Making huge amounts of log data accessible to learning algorithms provides the potential to acquire knowledge at unprecedented scale. Furthermore, the ability to learn from log data can enable effective machine learning even in systems where manual labeling of training data is not economically viable. Log data, however, provides only partial information -- "contextual-bandit feedback" -- limited to the particular actions taken by the system. The feedback for all the other actions the system could have taken is typically not known. This makes learning from log data fundamentally different from traditional supervised learning, where "correct" predictions together with a loss function provide full-information feedback.This project tackles the problem of Batch Learning from Bandit Feedback (BLBF) by developing principled learning methods and algorithms that can be trained with logs containing contextual-bandit feedback. First, the project develops the learning theory of BLBF, especially with respect to understanding the use and design of counterfactual risk estimators for BLBF. Second, the project derives new learning methods for BLBF. Past work has already demonstrated that Conditional Random Fields can be trained in the BLBF setting, and the project derives BLBF analogs of other learning methods as well. Third, the project derives scalable training algorithms for these BLBF methods to enable large-scale applications. And, finally, the project validates the methods with real-world data from operational systems.

日志数据是最普遍存在的可用数据形式之一，因为它可以从各种系统（例如，搜索引擎、推荐系统、广告投放平台）。让学习算法可以访问大量的日志数据，这为以前所未有的规模获取知识提供了可能。此外，从日志数据中学习的能力可以实现有效的机器学习，即使在人工标记训练数据在经济上不可行的系统中。然而，日志数据只提供了部分信息--“上下文强盗反馈”--仅限于系统所采取的特定操作。系统可能采取的所有其他动作的反馈通常是未知的。这使得从日志数据中学习与传统的监督学习有着根本的不同，传统的监督学习是通过“正确”的预测和损失函数提供完整的信息反馈。该项目通过开发原则性学习方法和算法来解决从Bandit反馈中批量学习（BLBF）的问题，这些方法和算法可以使用包含上下文-Bandit反馈的日志进行训练。首先，该项目发展了BLBF的学习理论，特别是在理解BLBF的反事实风险估计器的使用和设计方面。第二，该项目为BLBF衍生了新的学习方法。过去的工作已经证明，条件随机场可以在BLBF设置中进行训练，该项目还衍生出其他学习方法的BLBF类似物。第三，该项目为这些BLBF方法导出可扩展的训练算法，以实现大规模应用。最后，该项目用来自操作系统的真实数据验证了这些方法。