CIF: Small: Efficient Sequential Decision-Making and Inference in the Small Data Regime

从大数据中学习已经给推理和决策带来了革命性的变化，但有几个重要的应用属于小数据制度。在这种情况下，获取训练样本可能是昂贵、缓慢甚至危险的。因此，随着来自顺序样本的数据随着时间的推移而被接收，迫切需要能够进行顺序决策和推理。该项目开发了新的方法来提高顺序决策和推理的效率和准确性。它将通过在内容推荐系统、临床试验、分布式机器学习和超参数调整等应用程序中进行严格和有针对性的评估，展示预期结果的影响。研究成果将向广大学术和专业受众发表，并通过研究生和本科课程纳入教学课程。该项目将鼓励不同的学生群体参与研究。通过行业合作，这项研究的成果将迅速转化为实践。多臂强盗算法旨在最大化累积回报或在一组选择(称为ARM)中确定最佳选项，自然适合于涉及顺序决策的问题。然而，大多数关于多臂强盗算法的工作都假设各臂之间的报酬是独立的。该项目的目标是利用已知的潜在结构和臂之间的相关性来显著降低多臂盗贼算法的样本复杂度。特别是，研究人员的目标是为两个不同的框架设计样本高效的算法：i)结构化的强盗框架，其中奖励取决于共同的潜在特征向量；ii)新的相关强盗框架，其中来自ARM的奖励实现彼此关联。在这两个框架中，项目将导致算法的设计，以最大化累积回报(勘探-开采问题)，并尽可能快地确定最佳行动/武器(纯勘探问题)。这将通过一种新颖且易于推广的方法来实现，以说明关于结构或ARM之间的相关性的可用信息，以提高决策和推理的性能。该奖项反映了NSF的法定使命，并已通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Learning from big data has been revolutionizing inference and decision-making, and yet several important applications fall in the small data regime. In this regime, obtaining training samples can be expensive, slow or even hazardous. Thus, there is a critical need for enabling sequential decision-making and inference as data from sequential samples is received over time. This project develops new methods to improve the efficiency and accuracy of sequential decision-making and inference. It will demonstrate the impact of expected outcomes via rigorous and targeted evaluation in applications such as content recommendation systems, clinical trials, distributed machine learning, and hyperparameter tuning. The research outcomes will be published to broad academic and professional audiences and incorporated into teaching curricula via graduate and undergraduate courses. The project will encourage a diverse group of students to participate in research. Through industry partnerships, outcomes of this research will be transitioned quickly to practice.Multi-armed bandit algorithms, which aim to maximize the cumulative reward or identify the best option among a set of choices (referred to as arms), are naturally suited for problems involving sequential decision-making. However, most of the work on multi-armed bandit algorithms assumes independence of the rewards across arms. The objective of the proposed project is to exploit known latent structures and correlation between arms to drastically reduce the sample complexity of multi-armed bandit algorithms. In particular, the investigators aim to design sample-efficient algorithms for two different frameworks: i) the structured bandit framework, where the rewards depend on a common latent feature vector, and ii) a novel correlated bandit framework where reward realizations from arms are correlated with each other. In both frameworks, the project will result in the design of algorithms to maximize the cumulative reward (the exploration-exploitation problem) and to identify the best action/arm as fast as possible (the pure exploration problem). This will be done through a novel and easily generalizable approach to account for the available information on the structure or the correlation among arms to boost the performance of decision-making and inference.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.