Population-Based Evaluation of Artificial Intelligence for Mammography Prior to Widespread Clinical Translation

项目摘要 多种人工智能（AI）技术现在可用于自动解释 筛查性乳房X光检查这些人工智能技术有望提高筛查性能， 每年接受常规乳腺癌筛查的4000万美国妇女的结果。联邦 监管部门对新人工智能技术的批准只需要证明其准确性不低于现有技术。 计算机辅助检测系统在小型，回顾性读者研究，但其广泛的临床 翻译取决于更有力的基于人口的评价。具体而言，这些AI的影响 需要评估技术对实际患者结果的影响，包括它们是否导致改善 在一般筛查人群中检测出具有临床意义的癌症。AI的强大外部验证 迄今为止，乳房X线摄影筛查的算法受到使用单一机构数据集的限制， 代表整个目标人群，使用不公开的人工智能算法，与 放射科医生在丰富病例集中的表现，癌症诊断的随访时间有限，影响了地面 真值标签，以及对2D数字乳腺X射线摄影而非3D数字乳腺断层合成摄影（DBT）的评价 考试我们的研究目标是对五种市售AI进行比较评估 用于自动化DBT筛选解释的技术，克服了所有这些限制， 估计在美国人口水平上人工智能驱动的DBT筛查的长期益处，危害和成本。 具体来说，我们将1）使用一个集中的诚实代理，模型到数据的范例基础设施来执行一个 独立，外部验证五种领先的商业AI技术用于DBT筛查， 从8个不同的美国地区乳腺成像登记处前瞻性收集的数据; 2）分层AI vs.放射科医生在详细的女性、检查、放射科医生和肿瘤水平特征方面的表现， 有针对性的算法训练和细化工作，以确保AI算法的通用性; 3）探索 通过使用人工智能安全地对极有可能 阴性; 4）使用经验证的乳腺癌微观模拟模型来确定人群水平，长期 长期的健康益处，危害和与DBT筛查的AI技术相关的成本， 筛查工具，并作为第二个独立的读者，放射科医生的解释。我们建议的研究将 代表了对DBT筛选的深度学习算法的最客观和最严格的评估 在美国的诠释至今。我们的研究结果将提供急需的证据，以告知关键利益相关者 包括妇女、医生、支付者、行业合作伙伴和政策制定者， AI技术在广泛临床转化之前对DBT筛查的价值。

PROJECT SUMMARY Multiple artificial intelligence (AI) technologies are now commercially available for automated interpretation of screening mammography. These AI technologies hold promise for improving screening performance and outcomes for the 40 million U.S. women who undergo routine breast cancer screening each year. Federal regulatory approval of new AI technologies requires only a demonstration of non-inferior accuracy to existing computer-aided detection systems in small, retrospective reader studies, but their widespread clinical translation is contingent upon more robust population-based evaluation. Specifically, the impact of these AI technologies on actual patient outcomes needs to be assessed, including whether or not they lead to improved detection of clinically meaningful cancers in the general screening population. Robust external validation of AI algorithms for mammography screening has thus far been limited by use of single institution datasets not representative of the entire target population, use of AI algorithms that are not publicly available, comparison to radiologist performance in enriched case sets, limited follow-up time for cancer diagnoses influencing ground truth labels, and evaluation on 2D digital mammography rather than 3D digital breast tomosynthesis (DBT) exams. Our study objective is to conduct a comparative evaluation of five commercially available AI technologies for automated DBT screening interpretation that overcomes all of these limitations and then estimate the long-term benefits, harms, and costs of AI-driven DBT screening at the U.S. population level. Specifically, we will 1) use a centralized honest broker, model-to-data paradigm infrastructure to perform an independent, external validation of five leading commercial AI technologies for DBT screening using prospectively collected data obtained from eight diverse U.S. regional breast imaging registries; 2) stratify AI vs. radiologist performance on detailed woman-, exam-, radiologist-, and tumor-level characteristics to inform targeted algorithm training and refinement efforts to ensure generalizability of the AI algorithms; 3) explore targeted approaches for improving clinical workflow efficiency by using AI to safely triage exams highly likely to be negative; and 4) use a validated breast cancer microsimulation model to determine population-level, long- term health benefits, harms, and costs associated with AI technologies for DBT screening both as a standalone screening tool and as a second independent reader to radiologist interpretation. Our proposed study will represent the most objective and rigorous evaluation of deep learning algorithms for DBT screening interpretation in the U.S. to date. Our results will provide urgently needed evidence to inform key stakeholders including women, physicians, payers, industry partners, and policymakers regarding how to maximize the value of AI technologies for DBT screening prior to their widespread clinical translation.