Disciplinary Improvements: AI Readiness, Reproducibility, and FAIR: Connecting Computing and Domain Communities Across the ML Lifecycle

这个研究协调网络将在多学科社区基于公平数据原则建立更好的实践，重点关注三个主题：机器学习的公平、人工智能准备和重复性。选择这些专题是为了满足地球物理和计算机科学研究人员的迫切需要。解决的一个关键问题是，机器学习模型通常使用默认的“最佳”参数(例如预先训练的模型)传播，但缺乏关于模型训练、训练数据准备和永久识别符的文件。这阻碍了机器学习在地球科学中的科学重现性，并且经常低估了模型输出的方差。该项目将在地球物理学界以往的成功基础上，利用现有的网络在这一新的研究协调网络中建立关系，从而建立一个网络网络。将召集与机器学习有关的专家和亲和力小组，以了解新出现的最佳做法、可以利用哪些工具和资源，以及如何刺激实验，以量化数据的公平性与机器学习算法的容易和有效应用之间的关系，以及提高再现性。地球科学数据储存库将得到更好的装备，以支持其用户使用机器学习方法准备、存放、访问和重新使用数据。RCN还将制定一份路线图，作为社区领导的努力的指南，以便在需要将公平数据原则和开放科学应用于人工智能研究的领域集中注意力和资金。该项目将主办社区活动和工作组，收集所有三个主题的问题和实践。目标社区是地球物理数据档案提供商、研究人员自己创建的数据档案、机器学习科学家和从业者、高性能计算中心、围绕公平数据、大数据和人工智能的现有组织。该团队将协调社区活动并创建一系列报告，包括回顾和前瞻性指南。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

This research coordination network will build better practices based on the FAIR data principles in multiple disciplinary communities, focusing on three themes: FAIR in machine learning, AI readiness, and reproducibility. These themes were chosen to address the urgent needs of researchers in the geophysical and computer sciences. A key problem addressed is that machine learning models are often disseminated with default “best” parameters (e.g. pre-trained models), but lack documentation on model training, data preparation for training, and lacking permanent identifiers. This hinders scientific reproducibility of machine learning in the geosciences and often underestimates the variance in model outputs. The project will build on prior successes in the geophysical community and utilize existing networks to build relationships in this new research coordination network, thereby creating a network of networks. Experts and affinity groups related to machine learning will be convened to understand emerging best practices, which tools and resources to leverage, and how to stimulate experimentation that quantifies the relationship between the FAIRness of data and how easily and efficiently machine learning algorithms can be applied, as well as advancing reproducibility. Geosciences data repositories will be better equipped to support their users in preparation, deposit, access, and reuse of data using machine learning methods. The RCN will also develop a roadmap that will serve as a guide for community-led efforts to spotlight attention and funding in areas where an application of FAIR data principles and open science in AI research is needed.The project will host community events and working groups to gather issues and practices for all three themes. The target community is geophysical data archive providers, data archives created by researchers themselves, machine learning scientists and practitioners, high performance computing centers, existing organizations revolving around FAIR data, big data, and AI. The team will coordinate community activities and create a series of reports including both retrospective and forward-looking guidelines.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.