TOPIC 454 - SOFTWARE TO EVALUATE ARTIFICIAL INTELLIGENCE/MACHINE LEARNING MEDICAL DEVICES IN ONCOLOGY SETTINGS

主题 454 - 在肿瘤学环境中评估人工智能/机器学习医疗设备的软件

• 批准号: 10932590
• 负责人: JOSHUA MILLER
• 金额: $ 37.75万
• 依托单位: GRADIENT HEALTH INC
• 依托单位国家: 美国
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-15 至 2024-08-14
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10932590
• 关键词: Algorithms; Artificial Intelligence; Clinical; Communication Methods; Computer software; Consumption; Data; Data Set; Detection; Development; Devices; Elements; Evaluation; Feedback; Image; Instruction; Libraries; Machine Learning; Malignant Neoplasms; Manufacturer; Marketing; Medical Device; Medical Imaging; Methodology; Modeling; Oncology; Pathologic; Performance; Phase; Process; Qualifying; Radiology Specialty; Reporting; Secure; Source; Specific qualifier value; Test Result; Testing; Time; Validation; Vendor; artificial intelligence algorithm; experience; improved; interest; performance tests; prevent; privacy preservation; programs; radiological imaging; tool development

Supervised, Machine Learning based oncologic AI algorithms degrade over time and don’t generalize well. The FDA requires the conduct of standalone testing of radiologic, AI products to characterize their performance. Sourcing representative data with sufficient variability is time consuming, expensive, and often under-represent the variety of image quality experienced in clinical reality. FDA Reviewers also cannot compare new products to predicate devices. We propose a process for defining Reference Datasets and software allowing developers of imaging-based oncology AI products to test using the datasets. A Reference Dataset (R.D.) is an imaging dataset whose oncologic condition is confirmed by pathologic and/or radiographic confirmation. Objective 1 will be defining the rules and process to define R.D.s. Objective 2 will result in search and curation methodology to improve the extraction of oncologic data from our study library. Objective 3 will create an end-end workflow for the testing an imaging AI model against a RD curated. Finally, we will develop and submit a qualification plan for a Medical Device Development Tool to FDA in Objective 4. Machine Learning/AI can improve the detection and characterization of cancers from medical imaging but there are no common, ground-truth reference data to test and compare new AI algorithms.

基于监督的机器学习的肿瘤AI算法会随着时间的推移而退化，并且不能很好地推广。FDA要求对放射性人工智能产品进行独立测试，以表征其性能。获取具有足够可变性的代表性数据是耗时的、昂贵的，并且通常 不足以代表临床实际中所经历的各种图像质量。FDA审评员也不能将新产品与等同器械进行比较。我们提出了一个定义参考数据集和软件的过程，允许基于成像的肿瘤学AI产品的开发人员使用数据集进行测试。参考数据集（R. D.）是其肿瘤状况通过病理和/或组织学检查确认的成像数据集。 X光确认目标1将定义规则和流程，以定义R. D.目标2将导致搜索和策展方法，以提高我们的研究图书馆的肿瘤学数据的提取。目标3将创建一个端到端工作流程，用于根据研发策划测试成像AI模型。最后，我们将在目标4中制定并向FDA提交医疗器械开发工具的鉴定计划。机器学习/人工智能可以改善医学成像中癌症的检测和表征，但没有通用的地面参考数据来测试和比较新的人工智能算法。