Elements: CausalBench: A Cyberinfrastructure for Causal-Learning Benchmarking for Efficacy, Reproducibility, and Scientific Collaboration

要素：CausalBench：用于因果学习基准测试的网络基础设施，以实现有效性、可重复性和科学协作

• 批准号: 2311716
• 负责人: Kasim Candan
• 金额: $ 59.99万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-07-01 至 2026-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2311716&HistoricalAwards=false
• 关键词: Elements; CausalBench; Cyberinfrastructure; Causal; Learning

While we are witnessing the exceptional success of artificial intelligence (AI) and machine learning (ML) technologies in many applications, users are starting to notice a critical shortcoming of the current approaches: they are not causally grounded. While being relatively recent, causal learning aims to go far beyond conventional machine learning and is emerging as a vibrant field with new opportunities and challenges. Yet, advances in this field are hampered due to the lack of cyber-infrastructure platforms, with unified benchmarks data sets, algorithms, metrics, and evaluation service interfaces for causal learning. Reproducible science is possible only when the outcomes can be quantified and compared to other approaches and lack of reproducibility results in serious concerns on validity of published research. This can only be achieved through open platforms for data, algorithm, and model exchange and evaluation. Therefore, CausalBench, a transparent, fair, and easy-to-use evaluation platform, provides the key functionalities necessary to establish trust in causal learning’s innovation, collaboration, and critical applications, including public health and sustainability.CausalBench is a novel cyberinfrastructure of benchmarking data, algorithms, models, and metrics for causal learning, impacting the needs of a broad of scientific and engineering disciplines and sustain discovery across all fields. The cyberinfrastructure enables the advancement of research in causal learning by facilitating scientific collaboration in novel algorithms, datasets, and metrics and promotes scientific objectivity, reproducibility, fairness, and awareness of bias in causal learning research. CausalBench includes (1) an “ontology” for benchmarking to standardize the evaluation methodology, improve transparency, and promote collaboration to efficiently advance causal learning, (2) standard and convenient mechanisms for the community to contribute data and models such that disparate datasets can be integrated in a standard way, and (3) integrated evaluation standards that can help assess of algorithms for novel problems in the emerging field of causal learning with observational data. The project trains PhD students in the area of causal discovery and causally aware data management challenges via integrative, cross-disciplinary approaches and prepares future researchers with skills in data intensive AI and machine learning systems. The project further provides an excellent context for master’s, undergraduate, and K-12 students to be aware of AI and machine learning and their potential impacts on urgent societal challenges including public health, and sustainability.This award by the Office of Advanced Cyberinfrastructure is jointly supported by the Division of Information and Intelligent Systems within the Computer and Information Science and Engineering Directorate.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

虽然我们目睹了人工智能（AI）和机器学习（ML）技术在许多应用中取得的巨大成功，但用户开始注意到当前方法的一个关键缺点：它们没有因果关系。 虽然相对较新，但因果学习的目标远远超出了传统的机器学习，并正在成为一个充满活力的领域，带来新的机遇和挑战。然而，由于缺乏网络基础设施平台，缺乏统一的基准数据集，算法，指标和因果学习的评估服务接口，这一领域的进展受到阻碍。只有当结果可以量化并与其他方法进行比较时，可重复的科学才是可能的，而缺乏可重复性会导致对已发表研究有效性的严重担忧。这只能通过开放的数据、算法和模型交换和评估平台来实现。因此，Causal Bench是一个透明、公平和易于使用的评估平台，提供了建立对因果学习创新、协作和关键应用（包括公共卫生和可持续性）的信任所必需的关键功能。Causal Bench是一个新颖的网络基础设施，用于对因果学习的数据、算法、模型和指标进行基准测试，影响广泛的科学和工程学科的需求，并维持所有领域的发现。 网络基础设施通过促进新算法，数据集和指标的科学合作，促进因果学习研究的进步，并促进科学的客观性，可重复性，公平性和对因果学习研究中偏见的认识。Causal Bench包括（1）用于基准测试的“本体”，以标准化评估方法，提高透明度，并促进协作，以有效地推进因果学习，（2）标准和方便的机制，供社区贡献数据和模型，以便不同的数据集可以以标准的方式集成，和（3）综合评价标准，可以帮助评估算法的新问题，在新兴领域的因果学习与观测数据。 该项目通过综合的跨学科方法在因果发现和因果意识数据管理挑战领域培训博士生，并为未来的研究人员提供数据密集型人工智能和机器学习系统方面的技能。该项目还为硕士，本科和K-12学生提供了一个很好的环境，让他们了解人工智能和机器学习及其对紧急社会挑战的潜在影响，包括公共卫生，和可持续性。该奖项由高级网络基础设施办公室颁发，由计算机和信息科学与工程局信息和智能系统司共同支持。该奖项反映了NSF的法定使命，并已被认为是值得通过使用基金会的知识价值和更广泛的影响审查标准进行评估的支持。