Population-Based Evaluation of Artificial Intelligence for Mammography Prior to Widespread Clinical Translation

在广泛临床转化之前对乳腺 X 线摄影人工智能进行基于人群的评估

• 批准号: 10651842
• 负责人: CHRISTOPH I LEE
• 金额: $ 61.29万
• 依托单位: UNIVERSITY OF WASHINGTON
• 依托单位国家: 美国
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-07-01 至 2027-06-30
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10651842
• 关键词: 3-Dimensional; Academy; Address; Adopted; Algorithms; Artificial Intelligence; Biometry; Breast Cancer Detection; Breast Cancer Epidemiology; Breast Cancer Surveillance Consortium; Cancer Detection; Cancer Intervention and Surveillance Modeling Network; Characteristics; Clinical; Clinical effectiveness; Clinical/Radiologic; Data; Data Set; Detection; Digital Breast Tomosynthesis; Digital Mammography; Effectiveness; Ensure; Evaluation; Feedback; Funding; Future; Geography; Health Benefit; Health Services Research; Healthcare; Human; Image; Industry; Informatics; Infrastructure; Institution; Internal Medicine; International; Knowledge; Label; Link; Malignant Neoplasms; Mammographic screening; Mammography; Medicine; Modeling; Outcome; Patient-Focused Outcomes; Performance; Physicians; Policy Maker; Population; Privatization; Prospective Studies; Reader; Registries; Screening procedure; Target Populations; Technology; Technology Assessment; Testing; Time; Translations; Triage; United States; United States Food and Drug Administration; Update; Validation; Visual Perception; Woman; Women' s Group; algorithm training; artificial intelligence algorithm; breast imaging; cancer diagnosis; clinical practice; clinical translation; cohort; comparative; computer aided detection; cost; cost effectiveness; deep learning algorithm; deep neural network; detection platform; digital technology; follow-up; improved; improved outcome; industry partner; malignant breast neoplasm; models and simulation; mortality; multilevel analysis; neoplasm registry; novel; population based; prospective; radiologist; screening; tool; tumor

PROJECT SUMMARY Multiple artificial intelligence (AI) technologies are now commercially available for automated interpretation of screening mammography. These AI technologies hold promise for improving screening performance and outcomes for the 40 million U.S. women who undergo routine breast cancer screening each year. Federal regulatory approval of new AI technologies requires only a demonstration of non-inferior accuracy to existing computer-aided detection systems in small, retrospective reader studies, but their widespread clinical translation is contingent upon more robust population-based evaluation. Specifically, the impact of these AI technologies on actual patient outcomes needs to be assessed, including whether or not they lead to improved detection of clinically meaningful cancers in the general screening population. Robust external validation of AI algorithms for mammography screening has thus far been limited by use of single institution datasets not representative of the entire target population, use of AI algorithms that are not publicly available, comparison to radiologist performance in enriched case sets, limited follow-up time for cancer diagnoses influencing ground truth labels, and evaluation on 2D digital mammography rather than 3D digital breast tomosynthesis (DBT) exams. Our study objective is to conduct a comparative evaluation of five commercially available AI technologies for automated DBT screening interpretation that overcomes all of these limitations and then estimate the long-term benefits, harms, and costs of AI-driven DBT screening at the U.S. population level. Specifically, we will 1) use a centralized honest broker, model-to-data paradigm infrastructure to perform an independent, external validation of five leading commercial AI technologies for DBT screening using prospectively collected data obtained from eight diverse U.S. regional breast imaging registries; 2) stratify AI vs. radiologist performance on detailed woman-, exam-, radiologist-, and tumor-level characteristics to inform targeted algorithm training and refinement efforts to ensure generalizability of the AI algorithms; 3) explore targeted approaches for improving clinical workflow efficiency by using AI to safely triage exams highly likely to be negative; and 4) use a validated breast cancer microsimulation model to determine population-level, long- term health benefits, harms, and costs associated with AI technologies for DBT screening both as a standalone screening tool and as a second independent reader to radiologist interpretation. Our proposed study will represent the most objective and rigorous evaluation of deep learning algorithms for DBT screening interpretation in the U.S. to date. Our results will provide urgently needed evidence to inform key stakeholders including women, physicians, payers, industry partners, and policymakers regarding how to maximize the value of AI technologies for DBT screening prior to their widespread clinical translation.

项目概要 多种人工智能 (AI) 技术现已投入商业应用，可用于自动解释 筛查乳房X光检查。这些人工智能技术有望提高筛查性能和 每年接受常规乳腺癌筛查的 4000 万美国女性的结果。联邦 新人工智能技术的监管批准只需要证明其准确性不低于现有技术 小型回顾性读者研究中的计算机辅助检测系统，但其广泛的临床应用 翻译取决于更稳健的基于人群的评估。具体来说，这些人工智能的影响 需要评估技术对实际患者结果的影响，包括它们是否会导致改善 在一般筛查人群​​中检测出有临床意义的癌症。人工智能的强大外部验证 迄今为止，乳腺 X 线摄影筛查算法受到使用单一机构数据集的限制，而不是 代表整个目标人群，使用未公开的人工智能算法，与 放射科医生在丰富的病例集中的表现、癌症诊断的有限随访时间影响着地面 真实标签以及 2D 数字乳房 X 线摄影而非 3D 数字乳房断层合成 (DBT) 的评估 考试。我们的研究目标是对五种商用人工智能进行比较评估 自动 DBT 筛查解释技术克服了所有这些限制，然后 估计人工智能驱动的 DBT 筛查对美国人口的长期益处、危害和成本。 具体来说，我们将 1）使用集中式诚实代理、模型到数据范式基础设施来执行 使用五种领先的商业人工智能技术进行 DBT 筛查的独立外部验证 从美国八个不同​​的区域乳腺影像登记处前瞻性收集的数据； 2）人工智能分层 与放射科医生在详细的女性、检查、放射科医生和肿瘤水平特征上的表现对比 有针对性的算法训练和细化工作，以确保人工智能算法的通用性； 3）探索 通过使用人工智能安全地对检查进行分类来提高临床工作流程效率的有针对性的方法很可能 消极； 4) 使用经过验证的乳腺癌微观模拟模型来确定人群水平、长期 与用于 DBT 筛查的人工智能技术相关的术语健康益处、危害和成本（作为独立的技术） 筛查工具并作为放射科医生解释的第二个独立读者。我们提出的研究将 代表了对 DBT 筛查深度学习算法最客观、最严格的评估 迄今为止美国的解释。我们的结果将提供迫切需要的证据来告知主要利益相关者 包括妇女、医生、付款人、行业合作伙伴和政策制定者，讨论如何最大限度地提高 AI 技术在广泛临床转化之前对 DBT 筛查的价值。