Developing FAIR practices for cloud-enabled AI deployment for prospective testing

为基于云的人工智能部署制定公平实践以进行前瞻性测试

• 批准号: 10827803
• 负责人: Rima Arnaout
• 金额: $ 24.17万
• 依托单位: UNIVERSITY OF CALIFORNIA, SAN FRANCISCO
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-08 至 2025-03-31
• 项目状态: 未结题
• 来源: https://reporter.nih.gov/project-details/10827803
• 关键词: Algorithms; Artificial Intelligence; Benchmarking; Biometry; Birth; Cardiovascular system; Clinic; Clinical; Cloud Computing; Cloud Service; Code; Collaborations; Collection; Communities; Complex; Computer software; Computers; Congenital Abnormality; Data; Data Science; Data Scientist; Dedications; Democracy; Detection; Devices; Diagnosis; Documentation; Early Diagnosis; Early treatment; Echocardiography; Environment; Evaluation; Feasibility Studies; Feedback; Fetal Heart; Friends; Goals; Guidelines; Healthcare; Heart; Heart Abnormalities; Image; Image Analysis; Informatics; Institution; Instruction; Lesion; Life; Literature; Machine Learning; Medical Device; Medical Imaging; Methods; Mind; Modeling; Morbidity - disease rate; Netherlands; Outcome; Parents; Patients; Performance; Pregnant Women; Provider; Publishing; Recommendation; Reproducibility; Research; Research Personnel; Resources; Risk; Secure; Specialist; Speed; Surveys; Testing; Time; Translating; Ultrasonography; Validation; Work; biomedical imaging; clinical center; clinical imaging; clinically relevant; cloud platform; cloud storage; community center; community clinic; comorbidity; congenital heart disorder; cost; deep learning; deep learning algorithm; deep learning model; detector; disease diagnosis; experimental study; feasibility testing; fetal; health care settings; heart imaging; implementation research; improved; innovative technologies; insight; interoperability; mortality; multidisciplinary; novel; open source; open source tool; parent grant; prenatal; prevent; prospective; prospective test; repaired; screening; service providers; statistics; theories; tool; ultrasound; usability; user-friendly; web app

Supplemental Project Summary Objective — The goal of the parent proposal is to develop and optimize novel deep learning (DL) approaches to improve detection of congenital heart disease (CHD). We are using DL and related methods to extract diagnosis, biometric characterizations, and other information from fetal ultrasound imaging. Notably, this work includes retrospective evaluation in an imaging collection spanning two decades, tens of thousands of patients, and several clinical centers across a range of healthcare settings. Background — Despite clear benefits to prenatal detection of CHD and an ability for fetal ultrasound to detect over 90% of CHD lesions in theory, in practice the fetal CHD detection is closer to 50%. Prior literature suggests a key cause of this startling diagnosis gap is suboptimal acquisition and interpretation of fetal heart images. Preliminary Studies — Our multi-disciplinary team in CHD and data science has successfully used DL to distinguish normal hearts from those with complex CHD with an AUC of 0.99. Further retrospective validation shows our model to be an anomaly detector appropriate for screening and has generated novel insights into study quality and completeness that can improve clinical guidelines. The next logical step is prospective testing with a workflow robust enough for deployment in the community. We have developed clinical feasibility testing partners for cloud deployment, performed technical testing, and secured institutional approvals. Goals of Supplement — Prospective multi-center testing for DL algorithms seems well-suited to cloud deployment. However, determining how best to optimize cloud platforms with researchers, clinicians, and prospective testing in mind is an open question. Whether the cloud can be used to enable integration with, or testing aboard, end-user medical devices (‘edge devices’) is also unclear. Our goal in this supplement is to test these approaches. Aims — (1) to develop a workflow for hosting deep learning models on the cloud (users can send data to the cloud for inference directly from edge devices or via e.g. web application). Importantly, our approach is Findable, Available, Interoperable, and Reusable (FAIR), by leveraging tools offered across cloud service providers; being open-source, well-documented and well-covered by unit tests; being version-controlled and complete (“containerized”); and being user-friendly for biomedical researchers and clinical partners to use and re-use for different DL models. (2) We will perform feasibility testing in community fetal ultrasound clinics. Environment and Impact — This work is supported in an outstanding environment at the crossroads of data science, cardiovascular and fetal imaging, and translational informatics. Our testing partners span healthcare settings and the world, forcing our workflows to be robust. We will publish our workflow, software container, and documentation for the research community, as well as workflow and best practices for bringing the cloud to edge devices. The work proposed will provide valuable tools and insight into how best to use cloud services for rigorous and robust prospective testing of DL algorithms for biomedical imaging.

补充项目摘要 目标-母提案的目标是开发和优化新型深度学习（DL）方法 以提高先天性心脏病（CHD）的检出率。我们正在使用DL和相关方法来提取 诊断、生物特征和来自胎儿超声成像的其他信息。值得注意的是，这项工作 包括回顾性评价的成像收集跨越二十年，成千上万的病人， 以及多个医疗机构的临床中心。背景-尽管有明显的好处， 产前检测CHD和胎儿超声检测CHD病变的能力在理论上超过90%， 实践中胎儿CHD检出率接近50%。先前的文献表明，这一惊人的关键原因 诊断差距是胎儿心脏图像的次优采集和解释。初步研究-我们的 CHD和数据科学领域的多学科团队已成功使用DL区分正常心脏和 复杂CHD患者的AUC为0.99。进一步的回顾性验证表明，我们的模型是一个 异常检测器适用于筛选，并对研究质量产生了新的见解， 完整性，可以改善临床指南。下一个合乎逻辑的步骤是使用工作流进行前瞻性测试 足够强大，可以在社区中部署。我们已经开发了临床可行性测试合作伙伴， 云部署，执行技术测试，并获得机构批准。补充的目的- DL算法的前瞻性多中心测试似乎非常适合云部署。然而，在这方面， 根据研究人员、临床医生和前瞻性测试，确定如何最好地优化云平台 是个悬而未决的问题云是否可用于实现与最终用户的集成或在其上进行测试 医疗设备（“边缘设备”）也不清楚。我们在本补充中的目标是测试这些方法。 目标-（1）开发一个在云上托管深度学习模型的工作流程（用户可以将数据发送到 用于直接从边缘设备或经由例如web应用进行推断的云）。重要的是，我们的方法是 通过利用跨云服务提供的工具，实现可查找、可用、互操作和可重用（FAIR） 提供者;是开源的，有良好的文档记录，并被单元测试覆盖;是版本控制的， 完整（“集装箱化”）;便于生物医学研究人员和临床合作伙伴使用， 重复使用不同的DL模型。(2)我们将在社区胎儿超声诊所进行可行性测试。 环境和影响-这项工作是在一个出色的环境在数据的十字路口支持 科学，心血管和胎儿成像，以及翻译信息学。我们的测试合作伙伴涵盖医疗保健领域 环境和世界，迫使我们的工作流程变得强大。我们将发布我们的工作流程，软件容器， 和文档，以及将云计算引入 边缘设备。建议的工作将提供有价值的工具，并深入了解如何最好地使用云服务， 用于生物医学成像的DL算法的严格和稳健的前瞻性测试。