RI: Small: Using and Gathering Data for Efficient Batch Reinforcement Learning

想象一下，如果我们能在正确的时间为每个孩子提供正确的支持，帮助他们最好地学习，或者确保糖尿病患者得到最好的干预措施，帮助他们在家里管理他们的慢性病。不幸的是，这种个性化是昂贵的。更具可扩展性的计算机化方法可能缺乏提供有效个性化所需的实时信息，或缺乏专门干预的能力。然而，更用户友好的软件工具的大量增加意味着现在可以在广泛的设置中进行这种有针对性的个性化。 这项研究将开发利用现有数据的新方法，并创建算法，以与常见系统的限制兼容的方式获取新数据。这项工作可以帮助在比目前受益于这些方法的更广泛的应用程序中实现个性化干预。该研究将特别关注教育和医疗保健等领域所面临的技术挑战。更具体地说，该研究将创建数据高效算法和统计估计器，以利用过去关于决策及其结果的数据集，以及获取新的批量数据，这些数据可能会导致更好的结果，以创建决策策略--从描述当前上下文的特征到特定决策或干预的映射。特别是，该项目将集中在开发新的算法，优化政策的数据效率，最小的假设下的统计界限，对他们的未来表现;绑定的好处，收集预算的额外数据;并且，受到最优实验设计的启发，创建用于构建非自适应策略的算法，这些策略可用于收集数据，然后可以利用这些数据来识别近最优决策策略该研究将集中在两种情况下，一个单一的决定是在一个特定的背景下，并在一个序列的决定作出的决定，并作出影响下一个观察到的背景（常见的顺序决策下的不确定性过程）。这个奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

Imagine if we could provide each child with the right support, at the right time, for helping them learn best, or to ensure a diabetes patient is being given the best interventions to help them manage their chronic condition over time at home. Unfortunately such personalization is expensive. More scalable computerized approaches can lack the real-time information needed to provide effective personalization, or the ability to specialize interventions. However, the huge rise in more user-friendly software tools means that it is now possible to do such targeted personalization in a broad array of settings. This research will develop new methods for leveraging existing data, and create algorithms to acquire new data in a way that is compatible with the limitations of common systems. This work could help enable personalized interventions across a much broader array of applications than is currently benefiting from such approaches. The research will be particularly focused on the technical challenges arising from areas like education and healthcare.More specifically, this research will create data efficient algorithms and statistical estimators for leveraging past datasets about decisions made and their outcomes, and for acquiring new batch data that might lead to better results to create decision policies-- mappings from features describing the current context to a particular decision or intervention. In particular, the project will center on developing new algorithms that optimize policies with data efficient, minimal assumption lower statistical bounds on their future performance; bound the benefit of gathering a budget of additional data; and, inspired by insights from optimal experimental design, create algorithms for constructing non-adaptive policies that can be used to gather data that then can be leveraged to identify a near-optimal decision policy. The research will focus on both settings where a single decision is made for a particular context, and where a sequence of decisions are made and the decisions made impact the next context observed (common in sequential decision making under uncertainty processes).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.