Disciplinary Improvements: AI Readiness, Reproducibility, and FAIR: Connecting Computing and Domain Communities Across the ML Lifecycle

学科改进：AI 就绪性、可重复性和公平性：跨 ML 生命周期连接计算和领域社区

• 批准号: 2226453
• 负责人: Christine Kirkpatrick
• 金额: $ 126万
• 依托单位: University of California-San Diego
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-08-15 至 2025-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2226453&HistoricalAwards=false
• 关键词: Disciplinary; Improvements; AI; Readiness; Reproducibility

This research coordination network will build better practices based on the FAIR data principles in multiple disciplinary communities, focusing on three themes: FAIR in machine learning, AI readiness, and reproducibility. These themes were chosen to address the urgent needs of researchers in the geophysical and computer sciences. A key problem addressed is that machine learning models are often disseminated with default “best” parameters (e.g. pre-trained models), but lack documentation on model training, data preparation for training, and lacking permanent identifiers. This hinders scientific reproducibility of machine learning in the geosciences and often underestimates the variance in model outputs. The project will build on prior successes in the geophysical community and utilize existing networks to build relationships in this new research coordination network, thereby creating a network of networks. Experts and affinity groups related to machine learning will be convened to understand emerging best practices, which tools and resources to leverage, and how to stimulate experimentation that quantifies the relationship between the FAIRness of data and how easily and efficiently machine learning algorithms can be applied, as well as advancing reproducibility. Geosciences data repositories will be better equipped to support their users in preparation, deposit, access, and reuse of data using machine learning methods. The RCN will also develop a roadmap that will serve as a guide for community-led efforts to spotlight attention and funding in areas where an application of FAIR data principles and open science in AI research is needed.The project will host community events and working groups to gather issues and practices for all three themes. The target community is geophysical data archive providers, data archives created by researchers themselves, machine learning scientists and practitioners, high performance computing centers, existing organizations revolving around FAIR data, big data, and AI. The team will coordinate community activities and create a series of reports including both retrospective and forward-looking guidelines.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该研究协调网络将基于FAIR数据原则在多个学科社区建立更好的实践，重点关注三个主题：机器学习中的FAIR，AI准备和可重复性。 选择这些主题是为了满足地球物理和计算机科学研究人员的迫切需要。解决的一个关键问题是，机器学习模型通常使用默认的“最佳”参数（例如预先训练的模型）进行传播，但缺乏关于模型训练的文档，训练数据准备，并且缺乏永久标识符。这阻碍了机器学习在地球科学中的科学再现性，并且往往低估了模型输出的方差。 该项目将借鉴地球物理界以往的成功经验，利用现有网络在这一新的研究协调网络中建立关系，从而建立一个网络的网络。与机器学习相关的专家和亲和团体将被召集起来，以了解新兴的最佳实践，利用哪些工具和资源，以及如何刺激实验，量化数据的公平性与机器学习算法的应用之间的关系，以及提高可重复性。 地球科学数据储存库将得到更好的装备，以支持其用户使用机器学习方法准备、存款、访问和重新使用数据。RCN还将制定一个路线图，作为社区主导的努力的指南，在需要在人工智能研究中应用FAIR数据原则和开放科学的领域引起关注和资助。该项目将举办社区活动和工作组，收集所有三个主题的问题和实践。目标社区是地球物理数据存档提供商，研究人员自己创建的数据存档，机器学习科学家和从业者，高性能计算中心，围绕FAIR数据，大数据和AI的现有组织。该团队将协调社群活动，并创建一系列报告，包括回顾性和前瞻性的指导方针。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估的支持。