Collaborative Research: RI: Small: Foundations of Few-Round Active Learning

有监督的机器学习已经得到了广泛的应用，经常获得最先进的性能。然而，这些算法依赖于标记的训练实例，这可能是获取具有挑战性的。标记的实例通常由人工完成，需要时间和金钱才能获得。主动学习通过识别最具信息量的注释实例，努力将标注成本降至最低。虽然主动学习技术在产生标签较少的高性能模型方面表现出了希望，但由于需要与注释器进行多轮交互，这可能很耗时或不可行，因此它们的应用仍然受到限制。该项目旨在推进主动学习算法，并在互动回合有限的情况下理解其基本能力。预计广泛的机器学习应用程序将从这项研究的结果中受益，减少与获得足够数据以训练准确模型相关的时间和成本。此外，该项目通过实践研究和学习活动吸引未被充分代表的少数族裔学生，开发关于资源高效型机器学习的课程模块，并通过广泛的在线主动学习教程向工业界和学术界传播我们的研究结果。该项目将启动对少数几轮主动学习的全面调查，学习者可以在有限的轮次内主动请求对特定数据点的反馈。为了实现这一点，该项目将交织两个算法任务：稳健的数据效用量化和适应性有限的规划。首先，研究人员将探索衡量未标记数据效用的方法，考虑到数据大小、潜在数据特征和下游学习任务。随后，该团队将开发优化数据效用指标的算法，同时在几轮主动学习环境中随着时间的推移改善指标的质量。该项目的调查结果将为解决少数轮主动学习所特有的一种新的探索-开发困境确立原则性方法，并提供对适应性在预算内学习中的作用的基本理解。最后，该项目将评估各种高影响机器学习应用程序的建议方法，包括自动驾驶、智能建筑、对话系统和生化工程。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

Supervised machine learning has found widespread application, often achieving state-of-the-art performance. However, these algorithms rely on labeled training instances, which can be challenging to acquire. Labeled instances are often done by humans and require time and money to obtain. Active Learning strives to minimize labeling costs by identifying the most informative instances for annotation. While Active Learning techniques have shown promise in producing high-performance models with fewer labels, their applications remain constrained due to the necessity for multiple interaction rounds with annotators, which can be time-consuming or infeasible. This project aims to advance Active Learning algorithms and understanding of their fundamental capabilities in scenarios with limited interaction rounds. A broad spectrum of machine learning applications is expected to benefit from the results of this research, reducing the time and cost associated with obtaining sufficient data for training accurate models. Additionally, this project engages underrepresented minority students through hands-on research and learning activities, develops course modules on resource-efficient machine learning, and disseminates our findings to industry and academia via an extensive online Active Learning tutorial.This project will launch a comprehensive investigation of few-round active learning, where the learner can actively request feedback on specific data points within a limited number of rounds. To achieve this, the project will interleave two algorithmic tasks: robust data utility quantification and planning with limited adaptivity. First, the investigators will explore methods to measure the utility of unlabeled data, taking into account data size, underlying data characteristics, and downstream learning tasks. Subsequently, the team will develop algorithms that optimize the data utility metric while simultaneously improving the metric's quality over time in a few-round active learning setting. The project findings will establish principled approaches for addressing a novel exploration-exploitation dilemma specific to few-round active learning and provide a fundamental understanding of adaptivity's role in budgeted learning. Finally, the project will evaluate the proposed approaches across various high-impact machine learning applications, including autonomous driving, smart buildings, dialog systems, and biochemical engineering.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.