POSE: Phase I: Tuitus - A sustainable, inclusive, open ecosystem for Natural Hazards Engineering

该项目由开放源代码生态系统之路（POSE）资助，旨在利用开放源代码开发的力量，为国家和社会重要问题创造新的技术解决方案。洪水、山体滑坡、地震、龙卷风和野火等自然灾害威胁着美国57%以上的国家基础设施。对这些复杂的自然灾害进行建模推动了高性能计算、多尺度建模、现场可视化、大数据分析和机器学习的前沿。科学软件代码通常是由大学研究人员和研究生以特别的方式开发的，他们很少接触软件开发原则和实践。这些科学代码的健壮性和开源性对于国家、人民和环境的安全至关重要。该项目建立了Tuitus基金会（拉丁语为“保护或照顾”），这是一个可持续的，包容性的，开放的自然灾害工程科学规范生态系统。该项目的创新之处在于创建了一个统一的界面，支持分布式计算环境中的集成测试、研究再现性和工作流出处。该项目支持为高性能计算（HPC）环境构建可扩展的自动化测试和工作流管理。该项目通过建立董事会和支持NHE中社区驱动的软件开发来解决围绕建立和维持社区的社会问题。该项目创建了一个包容性的社区，通过黑客马拉松和Google Summer of Code为入门提供指导和培训，并支持新开发人员。通过将经过全面测试的HPC软件自动部署到NSF资助的DesignSafe Cyberinfrastructure，该项目通过立即访问强大的NHE代码，影响了更广泛的6，000名NHE研究人员社区。NHE中的开源代码将对决策过程和区域范围的风险评估产生重大影响。Tuitus基金会的HPC持续集成/持续交付（CI/CD）服务与DesignSafe的集成为NHE的开源代码交付带来了范式转变，改善了重大自然灾害事件期间的现实政策决策。Tuitus基金会采用整体方法来解决开源生态系统（OSE）中的关键技术挑战，为HPC和异构硬件架构提供标准应用程序编程接口（API），并支持HPC、机器学习、数据库模式、笔记本电脑和可视化等广泛应用。该项目是首次尝试将HPC规模的CI测试与多作业管道的自然灾害工程规范相结合。Tapis框架提供可编程接口，使开发人员能够在HPC和云资源上自动执行数据管理和作业。Tapis OAuth2提供了一个单独的安全层来管理用户凭证，允许Tapis代表用户访问许多不同的存储和计算资源。采用Tapis API的HPC级CI/CD可在分布式计算环境和未来的亿级系统上测试极端规模的问题。该项目通过跟踪工作流和打包运行时环境，在Jobyter笔记本中提供工作流出处，从而提高研究的可重复性。新的CI开发将文学编程和HPC测试结合在一起，提供急需的鲁棒性，以提高工程决策的可信度。该奖项反映了NSF的法定使命，并通过使用基金会的知识价值和更广泛的影响审查标准进行评估，被认为值得支持。

This project is funded by Pathways to Enable Open-Source Ecosystems (POSE) which seeks to harness the power of open-source development for the creation of new technology solutions to problems of national and societal importance. Natural hazards, such as floods, landslides, earthquakes, tornadoes, and wildfires, threaten more than 57% of the US national infrastructure. Modeling these complex natural hazards pushes the frontiers of high-performance computing, multi-scale modeling, in-situ visualization, big-data analysis, and machine learning. Scientific software codes are often developed in an ad hoc manner by university researchers and graduate students with very little exposure to software development principles and practices. The robustness and open-source nature of these scientific codes are essential for the safety and security of the nation, its people, and the environment. The project establishes the Tuitus Foundation (Latin for “to protect or care for”), a sustainable, inclusive, open ecosystem of scientific codes for Natural Hazard Engineering (NHE). The project’s novelties are creating a unified interface that supports integration testing, research reproducibility, and workflow provenance in distributed computing environments. The project supports building scalable, automated testing and workflow management for high-performance computing (HPC) environments. The project tackles the social issues around building and sustaining a community by establishing a Governing Board of Directors and supporting community-driven software development in NHE. The project creates an inclusive community by offering guidelines and training for onboarding and supporting new developers through hackathons and Google Summer of Code. By auto-deploying thoroughly tested HPC software to the NSF-funded DesignSafe Cyberinfrastructure, the project impacts the broader community of 6,000 NHE researchers by making robust NHE codes immediately accessible. Open-source codes in NHE will significantly impact the decision-making process and regional-scale risk assessment. The integration of Tuitus Foundation’s HPC Continuous Integration/Continuous Delivery (CI/CD) service with DesignSafe creates a paradigm shift in open-source code delivery in NHE, improving real-world policy decisions during critical natural hazard events. The Tuitus Foundation adopts a holistic approach to address the critical technical challenges in the Open-Source Ecosystem (OSE) by offering a standard Application Programming Interface (API) for HPC and heterogenous hardware architectures and supporting a wide range of applications from HPC, machine learning, database schemas, Jupyter notebooks, and visualizations. This project is the first attempt to integrate HPC-scale CI testing with natural hazard engineering codes for multi-job pipelines. The Tapis framework provides programmable interfaces that enable developers to automate data management and job execution on HPC and cloud resources. Tapis OAuth2 provides a separate security layer for managing user credentials, allowing Tapis to access many different storage and computing resources on behalf of the user. HPC-scale CI/CD with theTapis API enables testing for extreme-scale problems on distributed computing environments and future exascale systems. The project offers workflow provenance in Jupyter notebooks by tracking workflows and packaging runtime environment, thus enhancing research reproducibility. The novel CI developments bring together literate programming and HPC testing to provide the much-needed robustness to improve trustworthiness in engineering decision-making.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.