Trustworthy Reinforcement Learning for Online Decision Making

该研究项目将推进现代强化学习在线决策问题的前沿。强化学习处理的是智能代理如何在不确定的环境中采取行动，以最大化累积奖励。它在不同的商业和科学领域取得了惊人的成功。然而，很少有人关注强化学习的可信方面。该项目将研究重要在线决策任务中可靠问题的不同方面，如稳健性、公平性、因果关系和可解释性。这项研究的结果将有益于许多不同的领域，如统计学、机器学习、运筹学、市场营销、经济学和金融学。将开发开源软件，为应用研究人员提供尖端工具。该项目将招收学生，特别是那些来自未被代表群体的学生参与研究，并将开发有关统计强化学习和决策制定的新课程。本研究将聚焦于三种相互关联的可信赖强化学习方法，用于在线决策问题：动态定价、动态分类选择和双边市场的匹配。鲁棒性、公平性、因果关系和可解释性问题将在这些决策任务中得到解决，这将推进现有强化学习算法中使用的探索技术。该项目将开发新的理论工具来分析这些现代强化学习算法的统计特性。遗憾上界和匹配下界将深入研究。在线决策的一个重要目标是根据上下文信息和与环境的历史交互，确定一个使总体收益最大化的最优策略。由于这些问题的复杂性，人们对可靠的工具有很高的需求，以便在各种环境中学习最优的个性化策略。从这项研究中获得的知识将有助于在线拍卖和其他复杂的市场设计问题的学习。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This research project will advance the frontiers of modern reinforcement learning for online decision-making problems. Reinforcement learning deals with how intelligent agents ought to take actions in an uncertain environment in order to maximize the cumulative reward. It has achieved phenomenal success in diverse business and scientific fields. However, less attention has been paid to the trustworthy aspects of reinforcement learning. This project will investigate different aspects of trustworthy issues like robustness, fairness, causality, and explainability in important online decision-making tasks. The results of this research will benefit many different fields such as statistics, machine learning, operations research, marketing, economics, and finance. Open-source software will be developed to provide applied researchers with cutting-edge tools. The project will recruit students, especially those from unrepresented groups, to be involved in the research and will develop new courses on statistical reinforcement learning and decision making.This research project will focus on three interconnected trustworthy reinforcement learning methods for online decision-making problems: dynamic pricing, dynamic assortment selection, and matching in two-sided markets. Issues of robustness, fairness, causality, and explainability will be addressed in these decision-making tasks, which will advance the exploration techniques used in existing reinforcement learning algorithms. The project will develop new theoretical tools to analyze the statistical properties of these modern reinforcement learning algorithms. Regret upper bounds and matching lower bound will be thoroughly investigated. One important goal of online decision making is to identify an optimal policy that maximizes the overall gain, based on the contextual information and historical interactions with the environment. Due to the complex nature of such problems, there is a high demand for trustworthy tools for learning optimal personalized policy in various settings. The knowledge gained from this research will benefit learning in online auctions and other complex market design problems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.