CIF: Small: Efficient Sequential Decision-Making and Inference in the Small Data Regime

从大数据中学习已经彻底改变了推理和决策，但一些重要的应用程序属于小数据领域。在这种情况下，获取训练样本可能是昂贵的，缓慢的，甚至是危险的。因此，随着时间的推移接收来自顺序样本的数据，迫切需要实现顺序决策和推断。该项目开发了新的方法来提高顺序决策和推理的效率和准确性。它将通过对内容推荐系统、临床试验、分布式机器学习和超参数调整等应用进行严格和有针对性的评估，展示预期结果的影响。研究成果将发表给广泛的学术和专业观众，并通过研究生和本科生课程纳入教学课程。该项目将鼓励不同的学生群体参与研究。通过行业合作伙伴关系，这项研究的成果将迅速转化为实践。多臂强盗算法，其目的是最大限度地提高累积奖励或确定一组选择（称为武器）中的最佳选择，自然适合于涉及顺序决策的问题。然而，大多数关于多臂强盗算法的工作都假设跨臂的奖励是独立的。该项目的目标是利用已知的潜在结构和武器之间的相关性，大大降低多臂强盗算法的样本复杂度。特别是，研究人员的目标是为两个不同的框架设计样本有效的算法：i）结构化的强盗框架，其中奖励取决于一个共同的潜在特征向量，以及ii）一个新的相关强盗框架，其中来自武器的奖励实现相互关联。在这两个框架中，该项目将导致算法的设计，以最大限度地提高累积奖励（探索-利用问题），并尽快确定最佳动作/手臂（纯探索问题）。这将通过一种新颖且易于推广的方法来实现，以说明有关结构或武器之间相关性的可用信息，以提高决策和推理的性能。该奖项反映了NSF的法定使命，并被认为值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估来支持。

Learning from big data has been revolutionizing inference and decision-making, and yet several important applications fall in the small data regime. In this regime, obtaining training samples can be expensive, slow or even hazardous. Thus, there is a critical need for enabling sequential decision-making and inference as data from sequential samples is received over time. This project develops new methods to improve the efficiency and accuracy of sequential decision-making and inference. It will demonstrate the impact of expected outcomes via rigorous and targeted evaluation in applications such as content recommendation systems, clinical trials, distributed machine learning, and hyperparameter tuning. The research outcomes will be published to broad academic and professional audiences and incorporated into teaching curricula via graduate and undergraduate courses. The project will encourage a diverse group of students to participate in research. Through industry partnerships, outcomes of this research will be transitioned quickly to practice.Multi-armed bandit algorithms, which aim to maximize the cumulative reward or identify the best option among a set of choices (referred to as arms), are naturally suited for problems involving sequential decision-making. However, most of the work on multi-armed bandit algorithms assumes independence of the rewards across arms. The objective of the proposed project is to exploit known latent structures and correlation between arms to drastically reduce the sample complexity of multi-armed bandit algorithms. In particular, the investigators aim to design sample-efficient algorithms for two different frameworks: i) the structured bandit framework, where the rewards depend on a common latent feature vector, and ii) a novel correlated bandit framework where reward realizations from arms are correlated with each other. In both frameworks, the project will result in the design of algorithms to maximize the cumulative reward (the exploration-exploitation problem) and to identify the best action/arm as fast as possible (the pure exploration problem). This will be done through a novel and easily generalizable approach to account for the available information on the structure or the correlation among arms to boost the performance of decision-making and inference.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.