FW-HTF-RM: Measuring learning gains in man-machine assemblage when augmenting radiology work with artificial intelligence

未来的工作环境为人类与技术的伙伴关系提供了机会，人类与技术之间的和谐联系将产生前所未有的生产力提升。在这个人类-技术前沿的难题仍然存在-人类将被技术增强还是技术将被人类增强？这个项目克服了人类和机器作为独立实体的难题，而是将它们视为一个集合。作为和谐的人与技术联系的基础，这种组合需要学会协同配合。这种学习被称为集合学习，它对于人工智能（AI）在医疗保健中的应用非常重要，人工智能增强的诊断和治疗决策将对患者护理和结果产生直接而重大的影响。该项目还将确定在装配体之间共享学习的方式，从而可以创建连接装配体的集体群。该项目将创建一个新的学习模型，整合和衡量从个人学习到群体学习的概念。该项目将有助于展示一种共生学习组合，从而可以在不损失人类工作的情况下实现人工智能的预期生产力提高。尽管重点是放射学中的视觉认知任务，但该项目的经验教训可能适用于人类智能将通过机器智能增强的其他领域。最近的人类与机器竞争研究表明，联合收割机结合人类技术伙伴关系的组合比单个人类或机器更强大。在此基础上，该项目将把最先进的算法集成到放射学工作流程中。该项目将回答以下研究问题：Q1：如何开发组合，使人类技术的伙伴关系产生一个“良好的适合”的视觉为基础的认知导向的放射学任务？问题2：个人（放射科医生）和独立的机器学习模型中应该预先存在什么水平的培训，以使人类与技术的伙伴关系蓬勃发展？问题3：组装学习方法在哪些方面以及在多大程度上减少了错误，提高了准确性，缩短了周转时间，减少了疲劳，提高了自我效能和弹性？将进行一项严格的平衡试验，以评估放射科医生在使用和不使用组合的情况下解释图像。来自EHR系统的临床医生参与数据将被捕获和分析，沿着测试前和测试后的调查和访谈。对试验中的定量和定性数据进行深入和广泛的分析，将回答与组合中学习的学习收益、任务表现、情感和行为方面有关的问题。该项目采用科学技术研究，计算机科学，心理学和学习科学的观点，创造和研究可以在日常放射学工作中产生收益的组合。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的智力价值和更广泛的影响审查标准进行评估的支持。

The work setting of the future presents an opportunity for human-technology partnerships, where a harmonious connection between human-technology produces unprecedented productivity gains. A conundrum at this human-technology frontier remains - will humans be augmented by technology or will technology be augmented by humans? This project overcomes the conundrum of human and machine as separate entities and instead, treats them as an assemblage. As groundwork for the harmonious human-technology connection, this assemblage needs to learn to fit synergistically. This learning is called assemblage learning and it will be important for Artificial Intelligence (AI) applications in health care, where diagnostic and treatment decisions augmented by AI will have a direct and significant impact on patient care and outcomes. This project will also identify ways in which learning can be shared between assemblages, such that collective swarms of connected assemblages can be created. The project will create a new learning model that integrates and measures concepts from individuals learning to swarm learn. The project will help demonstrate a symbiotic learning assemblage, such that envisioned productivity gains from AI can be achieved without loss of human jobs. Even though the focus is on visual cognitive tasks in radiology, lessons from this project may be applicable to other domains where human intelligence will be augmented by machine intelligence.Recent studies of human versus machine competitions have demonstrated that assemblages that combine human-technology partnerships are stronger than individual humans or machines. By building on these, this project will integrate state-of-the-art algorithms into the radiology workflow. The project will answer the following research questions: Q1: How to develop assemblages, such that human-technology partnerships produce a "good fit" for visually based cognition-oriented tasks in radiology? Q2: What level of training should pre-exist in the individual human (radiologist) and independent machine learning model for human-technology partnerships to thrive? Q3: Which aspects and to what extent does an assemblage learning approach lead to reduced errors, improved accuracy, faster turn-around times, reduced fatigue, improved self-efficacy, and resilience? A rigorous counterbalanced trial will be performed to assess individual radiologists interpreting images with and without the assemblage. Data on clinician engagement from EHR systems will be captured and analyzed, along with pre-test and post-test surveys and interviews. Deep and wide analysis of the quantitative and qualitative data from the trial will answer questions related to learning gains, task performance, emotional as well as behavioral aspects of learning in an assemblage. The project employs perspectives from Science & Technology Studies, Computer Science, Psychology, and Learning Sciences, to create and study assemblages that can produce gains in routine radiology work.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.