PREMIERE: A PREdictive Model Index and Exchange REpository

PREMIERE：预测模型索引和交换存储库

• 批准号: 10597854
• 负责人: ALEX BUI
• 金额: $ 29.15万
• 依托单位: UNIVERSITY OF CALIFORNIA LOS ANGELES
• 依托单位国家: 美国
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-09-15 至 2023-05-31
• 项目状态: 已结题
• 来源: https://reporter.nih.gov/project-details/10597854
• 关键词: Address; Administrative Supplement; Artificial Intelligence; Attention; Awareness; Behavioral; Behavioral Research; Behavioral Sciences; Biomedical Research; Collaborations; Communication; Communities; Complex; Computational algorithm; Consent; Data; Data Element; Data Set; Decision Making; Development; Documentation; Educational workshop; Effectiveness; Electronic Health Record; Ethical Issues; Ethics; Extensible Markup Language; Funding; Future; Goals; Group Meetings; Health; Healthcare; Interview; Knowledge; Label; Language; Laws; Legal; Link; Machine Learning; Medical; Metadata; Methods; Modeling; Modernization; Outcome; Population; Prejudice; Privacy; Privatization; Process; Professional Ethics; Property; Proxy; Publishing; Qualitative Methods; Recommendation; Reproducibility; Research; Safety; Series; Shapes; Statistical Distributions; Structure; Surveys; System; Techniques; Training; Twin Multiple Birth; United States National Institutes of Health; Universities; Validation; Work; base; biomedical informatics; care delivery; deep learning; design; digital; disparity reduction; ethical legal social implication; generative adversarial network; improved; indexing; informant; insight; interest; learning algorithm; learning community; machine learning model; machine learning prediction; meetings; multidisciplinary; novel; predictive modeling; prototype; repository; response; sharing platform; simulation; social; symposium; tool; working group

PROJECT DESCRIPTION (ABSTRACT) The use of artificial intelligence (AI) continues to accelerate in biomedical and behavioral research, with the ultimate goals of informing and improving healthcare. While the technical advances thus far are significant, sev- eral concerns have been introduced recently regarding the (unintended) consequences of such techniques. For example, problems related to dataset bias can manifest in multiple ways, including the use of non-representative populations; continued propagation of unrecognized system and process prejudices; and equitable access. Given the potential downstream harm, ethical, legal, and social issues (ELSI) must now be integrated alongside the use of data and AI in biomedical and behavioral research and care delivery. However, best practices for ELSI and ethical AI (ETAI) have yet to fully emerge and there is no standard way of documenting ELSI/ETAI consid- erations in the development and use of predictive models. Building on our platform, the PREdictive Model Index and Exchange REpository (PREMIERE), the goal of this 1-year R01 supplement is to develop (meta)data to document and share information around ELSI/ETAI, directly linking such information as part of a shared predictive model. To focus our efforts, we address the growing use of synthetic datasets to train and validate machine learning (ML) models. Synthetic datasets reflect the underly- ing statistical properties of actual real-world datasets and are promoted as a way of protecting private information while enhancing the overall data availability (i.e., for training). The use of generative adversarial networks (GANs) is illustrative. But improperly simulated datasets can result in an algorithm learning incorrectly and/or exacerbat- ing existing dataset biases, raising complex ELSI questions. Using these questions as a motivating use case, this supplement has three specific aims: 1) to examine the ethics of using synthetic datasets, namely through key informant interviews; 2) to establish guidance for a computational checklist for AI/ML and ELSI, leveraging a broad community of stakeholders; and 3) to develop and implement this checklist as part of PREMIERE, demonstrating how ELSI-related information is shared as part of a ML model by extending the Predictive Model Markup Language (PMML). To achieve these aims, we established a new collaboration between UCLA and Penn State University (PSU) to bring together interdisciplinary experts in AI/ML, biomedical informatics, law, ethics, communication, and healthcare. Together, we will plan a series of workshops that convene national ex- perts who have already agreed to participate in this endeavor. The results of these meetings will be increased awareness around the use of synthetic datasets and their complexities; published recommendations around their use; and methods for documenting ELSI/ETAI in the context of predictive ML models.

项目描述（摘要） 人工智能（AI）在生物医学和行为研究中的应用不断加速， 告知和改善医疗保健的最终目标。虽然迄今为止的技术进步是显着的，但 最近人们对此类技术的（意外）后果产生了普遍的担忧。为了 例如，与数据集偏差相关的问题可以通过多种方式表现出来，包括使用非代表性 人口；未被认识到的系统和流程偏见的持续传播；和公平的机会。 鉴于潜在的下游危害，道德、法律和社会问题 (ELSI) 现在必须与 在生物医学和行为研究以及护理服务中使用数据和人工智能。然而，ELSI 的最佳实践 道德人工智能 (ETAI) 尚未完全出现，并且没有标准方法来记录 ELSI/ETAI 预测模型的开发和使用方面的进展。 以我们的平台预测模型索引和交易所存储库 (PREMIERE) 为基础，该目标 1 年 R01 补充是开发（元）数据来直接记录和共享有关 ELSI/ETAI 的信息 将这些信息链接起来作为共享预测模型的一部分。为了集中我们的努力，我们解决了日益增长的使用问题 用于训练和验证机器学习 (ML) 模型的合成数据集。综合数据集反映了潜在的 真实世界数据集的统计特性，并作为保护私人信息的一种方式进行推广 同时增强整体数据可用性（即用于培训）。生成对抗网络（GAN）的使用 是说明性的。但不正确的模拟数据集可能会导致算法学习不正确和/或加剧 现有数据集偏差，提出复杂的 ELSI 问题。使用这些问题作为激励用例， 该补充有三个具体目标：1）检查使用合成数据集的道德规范，即通过 关键知情人访谈； 2) 建立 AI/ML 和 ELSI 计算清单指南，利用 广泛的利益相关者群体； 3) 制定并实施此清单作为 PREMIERE 的一部分， 演示如何通过扩展预测模型来将 ELSI 相关信息作为 ML 模型的一部分进行共享 标记语言 (PMML)。为了实现这些目标，我们与加州大学洛杉矶分校 (UCLA) 和 宾夕法尼亚州立大学 (PSU) 汇聚人工智能/机器学习、生物医学信息学、法律、 道德、沟通和医疗保健。我们将共同策划一系列研讨会，召集全国前 已经同意参与这项工作的人。这些会议的成果将会增加 对合成数据集的使用及其复杂性的认识；围绕他们发表的建议 使用;以及在预测 ML 模型的背景下记录 ELSI/ETAI 的方法。