Elements: CausalBench: A Cyberinfrastructure for Causal-Learning Benchmarking for Efficacy, Reproducibility, and Scientific Collaboration

虽然我们看到人工智能（AI）和机器学习（ML）技术在许多应用中取得了非凡的成功，但用户开始注意到当前方法的一个关键缺点：它们没有因果关系基础。虽然相对较新，但因果学习的目标远远超出了传统的机器学习，并且正在成为一个充满新机遇和挑战的充满活力的领域。然而，由于缺乏网络基础设施平台，缺乏用于因果学习的统一基准数据集、算法、指标和评估服务接口，这一领域的进展受到阻碍。只有当结果可以量化并与其他方法进行比较时，可重复性科学才有可能实现，而缺乏可重复性会导致对已发表研究有效性的严重担忧。这只能通过开放的数据、算法和模型交换和评估平台来实现。因此，CausalBench是一个透明、公平且易于使用的评估平台，它提供了在因果学习的创新、协作和关键应用（包括公共卫生和可持续性）中建立信任所必需的关键功能。CausalBench是一种新型的网络基础设施，用于对因果学习的基准数据、算法、模型和指标进行测试，影响了广泛的科学和工程学科的需求，并在所有领域保持发现。网络基础设施通过促进在新算法、数据集和度量方面的科学合作，促进因果学习研究中的科学客观性、可重复性、公平性和偏见意识，从而推动因果学习研究的进步。CausalBench包括(1)一个基准测试的“本体”，用于标准化评估方法，提高透明度，促进协作，以有效地推进因果学习；(2)社区提供数据和模型的标准和便利机制，以便不同的数据集可以以标准的方式集成；(3)综合评估标准，可以帮助评估基于观测数据的因果学习新兴领域新问题的算法。该项目通过综合、跨学科的方法培养因果发现和因果感知数据管理挑战领域的博士生，并为未来的研究人员提供数据密集型人工智能和机器学习系统方面的技能。该项目还为硕士、本科生和K-12学生提供了一个很好的环境，让他们了解人工智能和机器学习，以及它们对公共卫生和可持续性等紧迫社会挑战的潜在影响。这项由先进网络基础设施办公室授予的合同，由计算机和信息科学与工程局下属的信息和智能系统司共同支持。该奖项反映了美国国家科学基金会的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

While we are witnessing the exceptional success of artificial intelligence (AI) and machine learning (ML) technologies in many applications, users are starting to notice a critical shortcoming of the current approaches: they are not causally grounded. While being relatively recent, causal learning aims to go far beyond conventional machine learning and is emerging as a vibrant field with new opportunities and challenges. Yet, advances in this field are hampered due to the lack of cyber-infrastructure platforms, with unified benchmarks data sets, algorithms, metrics, and evaluation service interfaces for causal learning. Reproducible science is possible only when the outcomes can be quantified and compared to other approaches and lack of reproducibility results in serious concerns on validity of published research. This can only be achieved through open platforms for data, algorithm, and model exchange and evaluation. Therefore, CausalBench, a transparent, fair, and easy-to-use evaluation platform, provides the key functionalities necessary to establish trust in causal learning’s innovation, collaboration, and critical applications, including public health and sustainability.CausalBench is a novel cyberinfrastructure of benchmarking data, algorithms, models, and metrics for causal learning, impacting the needs of a broad of scientific and engineering disciplines and sustain discovery across all fields. The cyberinfrastructure enables the advancement of research in causal learning by facilitating scientific collaboration in novel algorithms, datasets, and metrics and promotes scientific objectivity, reproducibility, fairness, and awareness of bias in causal learning research. CausalBench includes (1) an “ontology” for benchmarking to standardize the evaluation methodology, improve transparency, and promote collaboration to efficiently advance causal learning, (2) standard and convenient mechanisms for the community to contribute data and models such that disparate datasets can be integrated in a standard way, and (3) integrated evaluation standards that can help assess of algorithms for novel problems in the emerging field of causal learning with observational data. The project trains PhD students in the area of causal discovery and causally aware data management challenges via integrative, cross-disciplinary approaches and prepares future researchers with skills in data intensive AI and machine learning systems. The project further provides an excellent context for master’s, undergraduate, and K-12 students to be aware of AI and machine learning and their potential impacts on urgent societal challenges including public health, and sustainability.This award by the Office of Advanced Cyberinfrastructure is jointly supported by the Division of Information and Intelligent Systems within the Computer and Information Science and Engineering Directorate.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.