RI: Small: Using and Gathering Data for Efficient Batch Reinforcement Learning

想象一下，如果我们能在正确的时间为每个孩子提供正确的支持，帮助他们最好地学习，或者确保糖尿病患者得到最好的干预，帮助他们在家里长期控制自己的慢性疾病。不幸的是，这种个性化是昂贵的。更具可扩展性的计算机化方法可能缺乏提供有效个性化所需的实时信息，或者缺乏专门干预措施的能力。然而，越来越多的用户友好型软件工具的出现，意味着现在有可能在广泛的环境中进行这种有针对性的个性化设置。这项研究将开发利用现有数据的新方法，并创建算法，以与通用系统的局限性兼容的方式获取新数据。这项工作可以帮助在更广泛的应用中实现个性化干预，而不是目前从这些方法中受益。这项研究将特别关注教育和医疗保健等领域出现的技术挑战。更具体地说，这项研究将创建有效的数据算法和统计估计器，以利用过去关于决策及其结果的数据集，并获取可能导致更好结果的新批数据，以创建决策策略——从描述当前上下文的特征到特定决策或干预的映射。特别是，该项目将侧重于开发新的算法，以数据效率优化政策，对其未来性能的最小假设更低的统计界限；限制收集额外数据预算的好处；并且，受最优实验设计的启发，创建用于构建非自适应策略的算法，该策略可用于收集数据，然后可以利用这些数据来确定接近最优的决策策略。研究将集中在两种情况下，其中一个单一的决定是为一个特定的环境，并在一系列的决定，并作出的决定影响下一个上下文观察（在不确定性过程下的顺序决策中常见）。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

Imagine if we could provide each child with the right support, at the right time, for helping them learn best, or to ensure a diabetes patient is being given the best interventions to help them manage their chronic condition over time at home. Unfortunately such personalization is expensive. More scalable computerized approaches can lack the real-time information needed to provide effective personalization, or the ability to specialize interventions. However, the huge rise in more user-friendly software tools means that it is now possible to do such targeted personalization in a broad array of settings. This research will develop new methods for leveraging existing data, and create algorithms to acquire new data in a way that is compatible with the limitations of common systems. This work could help enable personalized interventions across a much broader array of applications than is currently benefiting from such approaches. The research will be particularly focused on the technical challenges arising from areas like education and healthcare.More specifically, this research will create data efficient algorithms and statistical estimators for leveraging past datasets about decisions made and their outcomes, and for acquiring new batch data that might lead to better results to create decision policies-- mappings from features describing the current context to a particular decision or intervention. In particular, the project will center on developing new algorithms that optimize policies with data efficient, minimal assumption lower statistical bounds on their future performance; bound the benefit of gathering a budget of additional data; and, inspired by insights from optimal experimental design, create algorithms for constructing non-adaptive policies that can be used to gather data that then can be leveraged to identify a near-optimal decision policy. The research will focus on both settings where a single decision is made for a particular context, and where a sequence of decisions are made and the decisions made impact the next context observed (common in sequential decision making under uncertainty processes).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.