Population-Based Evaluation of Artificial Intelligence for Mammography Prior to Widespread Clinical Translation

项目总结 多种人工智能(AI)技术现在可以在商业上用于自动解释 筛查乳房X光检查。这些人工智能技术有望提高筛选性能和 每年接受常规乳腺癌筛查的4000万美国女性的结果。联邦制 监管机构对新人工智能技术的批准只需证明其准确度不亚于现有的 计算机辅助检测系统在小型回溯性读者研究中的应用，但其广泛的临床应用 翻译取决于更有力的基于人口的评估。具体地说，这些人工智能的影响 需要评估技术对患者实际结果的影响，包括它们是否会带来改善 在普通筛查人群中检测有临床意义的癌症。强大的人工智能外部验证 到目前为止，用于乳房X光检查的算法受到使用单一机构数据集的限制，而不是 代表整个目标人群，使用未公开提供的人工智能算法，与 放射科医生在丰富的病例集上的表现，癌症诊断的有限随访时间影响了地面 2D数字乳房X光摄影而不是3D数字乳房断层合成(DBT)的真实性标签和评估 考试。我们的研究目标是对五种商业化的人工智能进行比较评估 用于自动DBT筛选解释的技术，克服了所有这些限制，然后 在美国人口水平上估计人工智能驱动的DBT筛查的长期好处、危害和成本。 具体地说，我们将1)使用集中的诚实代理、模型到数据范例基础设施来执行 对五项领先的商业人工智能技术进行独立的外部验证，用于DBT筛查 前瞻性收集从美国八个不同的地区乳房成像登记机构获得的数据；2)分层人工智能 与放射科医生在详细的女性、检查、放射科医生和肿瘤水平特征方面的表现进行比较 有针对性的算法训练和改进工作，以确保人工智能算法的泛化；3)探索 通过使用人工智能对检查进行安全分类来提高临床工作流程效率的有针对性的方法极有可能 为阴性；以及4)使用经过验证的乳腺癌微观模拟模型来确定人群水平、长期- 作为独立的DBT筛查，与人工智能技术相关的长期健康益处、危害和成本 作为筛查工具，并作为放射科医生解读的第二个独立读者。我们建议的研究将 代表对DBT筛选的深度学习算法进行最客观和严格的评估 到目前为止，在美国的解释。我们的结果将提供急需的证据，以告知关键利益相关者 包括女性、医生、付款人、行业合作伙伴和政策制定者，了解如何最大限度地提高 人工智能技术在广泛临床应用之前对DBT筛查的价值。

PROJECT SUMMARY Multiple artificial intelligence (AI) technologies are now commercially available for automated interpretation of screening mammography. These AI technologies hold promise for improving screening performance and outcomes for the 40 million U.S. women who undergo routine breast cancer screening each year. Federal regulatory approval of new AI technologies requires only a demonstration of non-inferior accuracy to existing computer-aided detection systems in small, retrospective reader studies, but their widespread clinical translation is contingent upon more robust population-based evaluation. Specifically, the impact of these AI technologies on actual patient outcomes needs to be assessed, including whether or not they lead to improved detection of clinically meaningful cancers in the general screening population. Robust external validation of AI algorithms for mammography screening has thus far been limited by use of single institution datasets not representative of the entire target population, use of AI algorithms that are not publicly available, comparison to radiologist performance in enriched case sets, limited follow-up time for cancer diagnoses influencing ground truth labels, and evaluation on 2D digital mammography rather than 3D digital breast tomosynthesis (DBT) exams. Our study objective is to conduct a comparative evaluation of five commercially available AI technologies for automated DBT screening interpretation that overcomes all of these limitations and then estimate the long-term benefits, harms, and costs of AI-driven DBT screening at the U.S. population level. Specifically, we will 1) use a centralized honest broker, model-to-data paradigm infrastructure to perform an independent, external validation of five leading commercial AI technologies for DBT screening using prospectively collected data obtained from eight diverse U.S. regional breast imaging registries; 2) stratify AI vs. radiologist performance on detailed woman-, exam-, radiologist-, and tumor-level characteristics to inform targeted algorithm training and refinement efforts to ensure generalizability of the AI algorithms; 3) explore targeted approaches for improving clinical workflow efficiency by using AI to safely triage exams highly likely to be negative; and 4) use a validated breast cancer microsimulation model to determine population-level, long- term health benefits, harms, and costs associated with AI technologies for DBT screening both as a standalone screening tool and as a second independent reader to radiologist interpretation. Our proposed study will represent the most objective and rigorous evaluation of deep learning algorithms for DBT screening interpretation in the U.S. to date. Our results will provide urgently needed evidence to inform key stakeholders including women, physicians, payers, industry partners, and policymakers regarding how to maximize the value of AI technologies for DBT screening prior to their widespread clinical translation.