RI: Small: Using and Gathering Data for Efficient Batch Reinforcement Learning

想象一下，如果我们可以在正确的时间为每个孩子提供正确的支持，帮助他们学习最好，或者确保糖尿病患者得到最好的干预，帮助他们随着时间的推移在家里管理他们的慢性病。不幸的是，这样的个性化是昂贵的。更具伸缩性的计算机化方法可能缺乏提供有效个性化所需的实时信息，或者缺乏专门干预的能力。然而，更加用户友好的软件工具的巨大增长意味着现在可以在广泛的环境中进行这种有针对性的个性化。这项研究将开发利用现有数据的新方法，并创建算法，以与常见系统的限制兼容的方式获取新数据。这项工作可以帮助在更广泛的应用程序中实现个性化干预，而不是目前受益于此类方法。这项研究将特别关注教育和医疗等领域带来的技术挑战。更具体地说，这项研究将创建数据高效的算法和统计估计器，以利用过去有关决策及其结果的数据集，并获取可能导致更好结果的新批数据，以创建决策政策--从描述当前背景的特征到特定决策或干预的映射。特别是，该项目将专注于开发新的算法，以数据高效、最少的假设降低未来性能的统计界限来优化策略；限制收集额外数据预算的好处；并受最佳实验设计的启发，创建用于构建非适应性策略的算法，这些策略可用于收集数据，然后可用于确定近乎最佳的决策策略。这项研究将集中在两个环境中，即针对特定背景做出单一决定，以及做出一系列决定并影响下一个观察到的背景(在不确定过程下的连续决策中常见)。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Imagine if we could provide each child with the right support, at the right time, for helping them learn best, or to ensure a diabetes patient is being given the best interventions to help them manage their chronic condition over time at home. Unfortunately such personalization is expensive. More scalable computerized approaches can lack the real-time information needed to provide effective personalization, or the ability to specialize interventions. However, the huge rise in more user-friendly software tools means that it is now possible to do such targeted personalization in a broad array of settings. This research will develop new methods for leveraging existing data, and create algorithms to acquire new data in a way that is compatible with the limitations of common systems. This work could help enable personalized interventions across a much broader array of applications than is currently benefiting from such approaches. The research will be particularly focused on the technical challenges arising from areas like education and healthcare.More specifically, this research will create data efficient algorithms and statistical estimators for leveraging past datasets about decisions made and their outcomes, and for acquiring new batch data that might lead to better results to create decision policies-- mappings from features describing the current context to a particular decision or intervention. In particular, the project will center on developing new algorithms that optimize policies with data efficient, minimal assumption lower statistical bounds on their future performance; bound the benefit of gathering a budget of additional data; and, inspired by insights from optimal experimental design, create algorithms for constructing non-adaptive policies that can be used to gather data that then can be leveraged to identify a near-optimal decision policy. The research will focus on both settings where a single decision is made for a particular context, and where a sequence of decisions are made and the decisions made impact the next context observed (common in sequential decision making under uncertainty processes).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.