CHS: Small: An Optimized Human-Machine Intelligence Framework for Single and Multi-Label Classification Tasks Through Active Learning

本项目的目标是通过主动学习为单标签和多标签分类问题开发一个优化的人机智能框架。重点是基于公民科学的网络人类系统，该系统收集物种观察信息用于生态学应用，但这些发现将适用于更广泛的领域。它将采用几种现有的主动学习技术来进行单标签和多标签分类，但在众包的背景下进行研究，特别是考虑以工人为中心的优化。创新之处在于系统地描述变量以建模人的因素，设计适当结合系统和以工人为中心的目标的优化模型，并发现创新的解决方案。该项目描述了一个迭代框架，它明智地雇佣人类工人收集标签，然后由监督机器算法使用这些标签进行智能预测。该框架通过将主动学习和人为因素建模策略恰当地结合起来，为单标签和多标签分类问题明智地选择任务和工作者，从而得出了它的原则。本文的主要贡献在于：1.通过人因建模考虑了与工人和任务相关的变量，建立了以工人为中心的优化模型。2.通过主动学习优化了单标签分类的人机智能模型。3.基于主动学习的多标签分类优化人机智能模型。4.人因估计、单标签和多标签任务分配的创新解决方案。5.公民科学领域的真实部署和严格评估。该项目可能会影响多个学科和领域的专家，如博物学家、生态学家、保护生物学家和公民科学家的作品。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

The goal of this project is to develop an optimized human-machine intelligence framework for single and multi-label classification problems through active learning. The focus is on citizen science based cyber-human systems that collect species observation information for ecological applications, but the findings will apply more broadly. It will adapt several existing active learning techniques for single and multi-label classification, but study them in the context of crowdsourcing, especially considering worker-centric optimization. The innovations lie in systematically characterizing variables to model human factors, designing optimization models that appropriately combine system and worker-centric goals, and discovering innovative solutions. The project describes an iterative framework that judiciously employs human workers to collect labels, which, in turn, are used by the supervised machine algorithms to make intelligent predictions. The framework derives its principles by appropriately combining active learning and human factors modeling strategies to judiciously select tasks and workers for single and multi-labels classification problems. The key intellectual contributions are: 1. Novel worker-centric optimization models considering variables pertinent to workers and tasks through human factors modeling. 2. Optimized human-machine intelligence models for single-label classification through active learning. 3. Optimized human-machine intelligence models for multi-labels classification through active learning. 4. Innovative solutions for human factors estimation, task assignment for single and multi-labels tasks. 5. Real world deployment and rigorous evaluation in the citizen science domain. The project is likely to impact multiple disciplines and domain experts, such as the works of naturalists, ecologists, conservation biologists and citizen scientists.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.