Trustworthy Reinforcement Learning: Inference, Reproducibility, and Adaptivity

近年来，我们看到人们对统计和机器学习在科学和公共政策中的各种问题领域的应用产生了浓厚的兴趣，从医疗保健到教育再到工程。统计方法的普及带来了新的基本问题。首先，大多数现代机器学习模型，包括深度学习和最近的深度强化学习，都是高维的，因此需要开发新的统计方法来推断这些模型。此外，这些模型中使用的数据集通常是按顺序收集的，因此，数据可能不满足独立和同分布的假设。例子包括自动驾驶汽车、在线广告、在线推荐系统和个性化治疗的数据。该项目旨在开发新的统计推断工具以及在独立非同分布环境中进行强化学习的计算高效方法，特别关注高维制度。该项目将为下一代统计学家和数据科学家的跨学科研究培训和专业发展提供大量机会。该项目包括两个关键目标，即统计科学和高维环境下的强化学习接口。第一个目标是在高维环境下进行强化学习的统计推断。数据独立且同分布的多维场景。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

In recent years, we have seen a surge of interest in the applications of statistics and machine learning to various problem domains in science and public policy, ranging from healthcare to education to engineering. This growth in the popularity of statistical methods comes with the new fundamental questions. First, most modern machine learning models, including deep learning and, more recently, deep reinforcement learning, are high-dimensional, thereby requiring development of novel statistical approaches for inference for such models. Furthermore, the datasets used in these models are often collected sequentially and, as a result, the data may not satisfy the assumptions of being independent and identically distributed. Examples include data for self-driving cars, online advertising, online recommendation systems, and personalized treatments. This project aims to develop novel statistical inference tools as well as computationally efficient approaches for reinforcement learning in independent non-identically distributed settings, with a particular focus on high-dimensional regimes. The project will offer numerous opportunities for interdisciplinary research training and professional development of the next generation of statisticians and data scientists.The project consists of the two key thrusts at the interface of statistical sciences and reinforcement learning in high-dimensional settings.The first thrust is geared toward statistical inference for reinforcement learning under the high-dimensional scenarios where the data are independent and identically distributed. The focus of the second thrust is to design computationally efficient algorithms, specifically focusing on online and offline reinforcement learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.