Elements: CausalBench: A Cyberinfrastructure for Causal-Learning Benchmarking for Efficacy, Reproducibility, and Scientific Collaboration

虽然我们见证了人工智能(AI)和机器学习(ML)技术在许多应用中的非凡成功，但用户开始注意到当前方法的一个严重缺陷：它们没有因果关系。虽然因果学习相对较新，但它的目标是远远超越传统的机器学习，并正在成为一个充满活力的领域，面临着新的机遇和挑战。然而，由于缺乏网络基础设施平台，缺乏统一的基准数据集、算法、指标和用于因果学习的评估服务接口，这一领域的进展受到阻碍。只有当结果可以量化并与其他方法进行比较时，可重复性科学才是可能的，而重复性的缺乏导致对已发表研究的有效性的严重担忧。这只能通过开放的数据、算法和模型交换和评估平台来实现。因此，一个透明、公平和易于使用的评估平台CausalBch提供了在因果学习的创新、协作和关键应用(包括公共卫生和可持续发展)中建立信任所需的关键功能。CausalBch是一个新型的网络基础设施，包含用于因果学习的基准数据、算法、模型和指标，影响到广泛的科学和工程学科的需求，并支持跨所有领域的发现。网络基础设施通过促进在新算法、数据集和指标方面的科学合作，促进因果学习研究的进步，并促进科学客观性、重复性、公平性和对因果学习研究中的偏见的认识。CausalBitch包括(1)用于基准测试的“本体”，以标准化评估方法，提高透明度，并促进协作以有效地推进因果学习；(2)标准和便利的机制，供社区提供数据和模型，以便以标准的方式整合不同的数据集；以及(3)集成的评估标准，可以帮助评估新兴的因果学习领域中新问题的算法与观测数据。该项目通过综合、跨学科的方法，在因果发现和因果意识数据管理挑战领域培训博士生，并为未来的研究人员培养数据密集型人工智能和机器学习系统方面的技能。该项目还为硕士、本科生和K-12年级的学生提供了一个很好的环境，让他们了解人工智能和机器学习及其对包括公共卫生和可持续发展在内的紧迫社会挑战的潜在影响。该奖项由高级网络基础设施办公室颁发，由计算机和信息科学与工程总监内的信息和智能系统司联合支持。该奖项反映了NSF的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。

While we are witnessing the exceptional success of artificial intelligence (AI) and machine learning (ML) technologies in many applications, users are starting to notice a critical shortcoming of the current approaches: they are not causally grounded. While being relatively recent, causal learning aims to go far beyond conventional machine learning and is emerging as a vibrant field with new opportunities and challenges. Yet, advances in this field are hampered due to the lack of cyber-infrastructure platforms, with unified benchmarks data sets, algorithms, metrics, and evaluation service interfaces for causal learning. Reproducible science is possible only when the outcomes can be quantified and compared to other approaches and lack of reproducibility results in serious concerns on validity of published research. This can only be achieved through open platforms for data, algorithm, and model exchange and evaluation. Therefore, CausalBench, a transparent, fair, and easy-to-use evaluation platform, provides the key functionalities necessary to establish trust in causal learning’s innovation, collaboration, and critical applications, including public health and sustainability.CausalBench is a novel cyberinfrastructure of benchmarking data, algorithms, models, and metrics for causal learning, impacting the needs of a broad of scientific and engineering disciplines and sustain discovery across all fields. The cyberinfrastructure enables the advancement of research in causal learning by facilitating scientific collaboration in novel algorithms, datasets, and metrics and promotes scientific objectivity, reproducibility, fairness, and awareness of bias in causal learning research. CausalBench includes (1) an “ontology” for benchmarking to standardize the evaluation methodology, improve transparency, and promote collaboration to efficiently advance causal learning, (2) standard and convenient mechanisms for the community to contribute data and models such that disparate datasets can be integrated in a standard way, and (3) integrated evaluation standards that can help assess of algorithms for novel problems in the emerging field of causal learning with observational data. The project trains PhD students in the area of causal discovery and causally aware data management challenges via integrative, cross-disciplinary approaches and prepares future researchers with skills in data intensive AI and machine learning systems. The project further provides an excellent context for master’s, undergraduate, and K-12 students to be aware of AI and machine learning and their potential impacts on urgent societal challenges including public health, and sustainability.This award by the Office of Advanced Cyberinfrastructure is jointly supported by the Division of Information and Intelligent Systems within the Computer and Information Science and Engineering Directorate.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.