FW-HTF-RM: Measuring learning gains in man-machine assemblage when augmenting radiology work with artificial intelligence

未来的工作环境为人类-技术伙伴关系提供了机会，人类-技术之间的和谐联系产生了前所未有的生产力提高。在这个人类与技术的前沿，一个难题仍然存在——是人类被技术所增强，还是技术被人类所增强？这个项目克服了人类和机器作为独立实体的难题，而是将它们视为一个组合。作为和谐的人与技术联系的基础，这一组合需要学会协同适应。这种学习被称为集合学习，它对于人工智能（AI）在医疗保健中的应用非常重要，人工智能增强的诊断和治疗决策将对患者的护理和结果产生直接而重大的影响。该项目还将确定在组合之间共享学习的方法，这样就可以创建连接组合的集体群。该项目将创建一种新的学习模式，整合并衡量从个体学习到群体学习的概念。该项目将有助于展示一种共生学习组合，这样就可以在不损失人类工作的情况下实现人工智能带来的生产率提高。尽管重点是放射学中的视觉认知任务，但该项目的经验教训可能适用于人类智能将被机器智能增强的其他领域。最近对人类与机器竞争的研究表明，结合人类技术伙伴关系的组合比单个人类或机器更强大。在此基础上，该项目将把最先进的算法集成到放射学工作流程中。该项目将回答以下研究问题：Q1：如何开发组合，使人类与技术的合作伙伴关系为放射学中基于视觉的认知导向任务产生“良好的契合”？问题2：为了使人类技术合作伙伴关系蓬勃发展，个体（放射科医生）和独立的机器学习模型应该预先存在什么水平的培训？Q3：集合学习方法在哪些方面以及在多大程度上减少了错误、提高了准确性、加快了周转时间、减少了疲劳、提高了自我效能和弹性？一个严格的平衡试验将被执行，以评估个别放射科医生解释图像与没有组合。将收集和分析来自电子病历系统的临床医生参与情况数据，以及测试前和测试后的调查和访谈。对试验中定量和定性数据进行深入和广泛的分析，将回答与学习收益、任务表现、情感和行为方面有关的问题。该项目采用科学与技术研究、计算机科学、心理学和学习科学的观点，创建和研究可在常规放射学工作中产生收益的组合。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

The work setting of the future presents an opportunity for human-technology partnerships, where a harmonious connection between human-technology produces unprecedented productivity gains. A conundrum at this human-technology frontier remains - will humans be augmented by technology or will technology be augmented by humans? This project overcomes the conundrum of human and machine as separate entities and instead, treats them as an assemblage. As groundwork for the harmonious human-technology connection, this assemblage needs to learn to fit synergistically. This learning is called assemblage learning and it will be important for Artificial Intelligence (AI) applications in health care, where diagnostic and treatment decisions augmented by AI will have a direct and significant impact on patient care and outcomes. This project will also identify ways in which learning can be shared between assemblages, such that collective swarms of connected assemblages can be created. The project will create a new learning model that integrates and measures concepts from individuals learning to swarm learn. The project will help demonstrate a symbiotic learning assemblage, such that envisioned productivity gains from AI can be achieved without loss of human jobs. Even though the focus is on visual cognitive tasks in radiology, lessons from this project may be applicable to other domains where human intelligence will be augmented by machine intelligence.Recent studies of human versus machine competitions have demonstrated that assemblages that combine human-technology partnerships are stronger than individual humans or machines. By building on these, this project will integrate state-of-the-art algorithms into the radiology workflow. The project will answer the following research questions: Q1: How to develop assemblages, such that human-technology partnerships produce a "good fit" for visually based cognition-oriented tasks in radiology? Q2: What level of training should pre-exist in the individual human (radiologist) and independent machine learning model for human-technology partnerships to thrive? Q3: Which aspects and to what extent does an assemblage learning approach lead to reduced errors, improved accuracy, faster turn-around times, reduced fatigue, improved self-efficacy, and resilience? A rigorous counterbalanced trial will be performed to assess individual radiologists interpreting images with and without the assemblage. Data on clinician engagement from EHR systems will be captured and analyzed, along with pre-test and post-test surveys and interviews. Deep and wide analysis of the quantitative and qualitative data from the trial will answer questions related to learning gains, task performance, emotional as well as behavioral aspects of learning in an assemblage. The project employs perspectives from Science & Technology Studies, Computer Science, Psychology, and Learning Sciences, to create and study assemblages that can produce gains in routine radiology work.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.